[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260619 21:14:34.862846 396 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.62:32921
I20260619 21:14:34.864235 396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 21:14:34.865387 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:14:34.901811 403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:14:34.903862 402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:14:34.901842 405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:14:36.220394 404 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1319 milliseconds
I20260619 21:14:36.220518 396 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260619 21:14:36.222009 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:14:36.222252 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:14:36.222465 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903676222440 us; error 0 us; skew 500 ppm
I20260619 21:14:36.223323 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:14:36.229408 396 webserver.cc:460] Webserver started at http://127.0.99.62:34019/ using document root <none> and password file <none>
I20260619 21:14:36.230283 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:14:36.230506 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:14:36.230978 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:14:36.238230 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/master-0-root/instance:
uuid: "5d773a50e1984526a6d2a8a5df258f5f"
format_stamp: "Formatted at 2026-06-19 21:14:36 on dist-test-slave-nlqf"
I20260619 21:14:36.248107 396 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.008s
I20260619 21:14:36.255138 411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:36.256543 396 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260619 21:14:36.256884 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/master-0-root
uuid: "5d773a50e1984526a6d2a8a5df258f5f"
format_stamp: "Formatted at 2026-06-19 21:14:36 on dist-test-slave-nlqf"
I20260619 21:14:36.257267 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:14:36.302587 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:14:36.304594 396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 21:14:36.305207 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:14:36.485661 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.62:32921
I20260619 21:14:36.485924 462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.62:32921 every 8 connection(s)
I20260619 21:14:36.497225 463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:36.532774 463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f: Bootstrap starting.
I20260619 21:14:36.542508 463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:36.545807 463 log.cc:824] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f: Log is configured to *not* fsync() on all Append() calls
I20260619 21:14:36.553516 463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f: No bootstrap required, opened a new log
I20260619 21:14:36.589473 463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d773a50e1984526a6d2a8a5df258f5f" member_type: VOTER }
I20260619 21:14:36.590256 463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:36.590654 463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d773a50e1984526a6d2a8a5df258f5f, State: Initialized, Role: FOLLOWER
I20260619 21:14:36.591722 463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d773a50e1984526a6d2a8a5df258f5f" member_type: VOTER }
I20260619 21:14:36.592373 463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 21:14:36.592722 463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 21:14:36.593222 463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:36.599251 463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d773a50e1984526a6d2a8a5df258f5f" member_type: VOTER }
I20260619 21:14:36.600229 463 leader_election.cc:302] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d773a50e1984526a6d2a8a5df258f5f; no voters:
I20260619 21:14:36.604084 463 leader_election.cc:288] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 21:14:36.604595 466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:36.612355 466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 1 LEADER]: Becoming Leader. State: Replica: 5d773a50e1984526a6d2a8a5df258f5f, State: Running, Role: LEADER
I20260619 21:14:36.613875 466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d773a50e1984526a6d2a8a5df258f5f" member_type: VOTER }
I20260619 21:14:36.616053 463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 21:14:36.637882 467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d773a50e1984526a6d2a8a5df258f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d773a50e1984526a6d2a8a5df258f5f" member_type: VOTER } }
I20260619 21:14:36.638908 467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [sys.catalog]: This master's current role is: LEADER
I20260619 21:14:36.637545 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d773a50e1984526a6d2a8a5df258f5f. Latest consensus state: current_term: 1 leader_uuid: "5d773a50e1984526a6d2a8a5df258f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d773a50e1984526a6d2a8a5df258f5f" member_type: VOTER } }
I20260619 21:14:36.641335 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [sys.catalog]: This master's current role is: LEADER
I20260619 21:14:36.646952 474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 21:14:36.672320 474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 21:14:36.687549 396 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260619 21:14:36.704174 474 catalog_manager.cc:1347] Generated new cluster ID: 727d14ffed9943b0a472e02a29cf5570
I20260619 21:14:36.704614 474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 21:14:36.731709 474 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 21:14:36.735617 474 catalog_manager.cc:1504] Loading token signing keys...
I20260619 21:14:36.754566 474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f: Generated new TSK 0
I20260619 21:14:36.755786 474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 21:14:36.824680 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:14:36.840251 485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:14:36.843775 486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:14:36.855523 396 server_base.cc:1034] running on GCE node
W20260619 21:14:36.856482 488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:14:36.857558 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:14:36.857758 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:14:36.857961 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903676857946 us; error 0 us; skew 500 ppm
I20260619 21:14:36.858588 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:14:36.866773 396 webserver.cc:460] Webserver started at http://127.0.99.1:43491/ using document root <none> and password file <none>
I20260619 21:14:36.867403 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:14:36.867601 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:14:36.867909 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:14:36.869251 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-0-root/instance:
uuid: "86439c50df6044b7be9eccfcf6066c1a"
format_stamp: "Formatted at 2026-06-19 21:14:36 on dist-test-slave-nlqf"
I20260619 21:14:36.878767 396 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.008s
I20260619 21:14:36.886296 493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:36.887356 396 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.005s
I20260619 21:14:36.887626 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-0-root
uuid: "86439c50df6044b7be9eccfcf6066c1a"
format_stamp: "Formatted at 2026-06-19 21:14:36 on dist-test-slave-nlqf"
I20260619 21:14:36.887920 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:14:36.919438 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:14:36.921504 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:14:36.927441 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:14:36.934235 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:14:36.934509 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:36.934863 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:14:36.935053 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.187546 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.1:40099
I20260619 21:14:37.187701 555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.1:40099 every 8 connection(s)
I20260619 21:14:37.197207 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:14:37.222595 560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:14:37.236287 561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:14:37.245597 396 server_base.cc:1034] running on GCE node
W20260619 21:14:37.247575 564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:14:37.249091 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:14:37.249413 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:14:37.249667 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903677249645 us; error 0 us; skew 500 ppm
I20260619 21:14:37.250705 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:14:37.254706 396 webserver.cc:460] Webserver started at http://127.0.99.2:44961/ using document root <none> and password file <none>
I20260619 21:14:37.255899 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:14:37.256215 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:14:37.256505 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:14:37.257781 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-1-root/instance:
uuid: "18e29b563e474386ac80d2afc78e3690"
format_stamp: "Formatted at 2026-06-19 21:14:37 on dist-test-slave-nlqf"
I20260619 21:14:37.260205 556 heartbeater.cc:347] Connected to a master server at 127.0.99.62:32921
I20260619 21:14:37.260790 556 heartbeater.cc:464] Registering TS with master...
I20260619 21:14:37.261904 556 heartbeater.cc:511] Master 127.0.99.62:32921 requested a full tablet report, sending...
I20260619 21:14:37.264163 396 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260619 21:14:37.265722 428 ts_manager.cc:194] Registered new tserver with Master: 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:37.268167 568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.269233 396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 21:14:37.269501 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-1-root
uuid: "18e29b563e474386ac80d2afc78e3690"
format_stamp: "Formatted at 2026-06-19 21:14:37 on dist-test-slave-nlqf"
I20260619 21:14:37.269780 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:14:37.269941 428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38216
I20260619 21:14:37.331064 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:14:37.332788 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:14:37.335713 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:14:37.339618 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:14:37.339851 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.340090 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:14:37.340252 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.596339 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.2:33541
I20260619 21:14:37.597007 630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.2:33541 every 8 connection(s)
I20260619 21:14:37.610638 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:14:37.630213 634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:14:37.632478 635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:14:37.646528 631 heartbeater.cc:347] Connected to a master server at 127.0.99.62:32921
I20260619 21:14:37.647172 631 heartbeater.cc:464] Registering TS with master...
I20260619 21:14:37.648754 631 heartbeater.cc:511] Master 127.0.99.62:32921 requested a full tablet report, sending...
I20260619 21:14:37.652570 428 ts_manager.cc:194] Registered new tserver with Master: 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:37.656173 428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38228
I20260619 21:14:37.657104 396 server_base.cc:1034] running on GCE node
W20260619 21:14:37.659749 637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:14:37.661968 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:14:37.662235 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:14:37.662434 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903677662417 us; error 0 us; skew 500 ppm
I20260619 21:14:37.663136 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:14:37.665994 396 webserver.cc:460] Webserver started at http://127.0.99.3:38629/ using document root <none> and password file <none>
I20260619 21:14:37.666659 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:14:37.666915 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:14:37.667232 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:14:37.671480 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-2-root/instance:
uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
format_stamp: "Formatted at 2026-06-19 21:14:37 on dist-test-slave-nlqf"
I20260619 21:14:37.678050 396 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260619 21:14:37.682550 642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.683569 396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260619 21:14:37.683885 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-2-root
uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
format_stamp: "Formatted at 2026-06-19 21:14:37 on dist-test-slave-nlqf"
I20260619 21:14:37.684229 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781903674708472-396-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:14:37.723510 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:14:37.725364 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:14:37.728192 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:14:37.732435 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:14:37.732721 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.733021 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:14:37.733232 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:14:37.988683 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.3:37523
I20260619 21:14:37.989423 704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.3:37523 every 8 connection(s)
I20260619 21:14:38.028339 705 heartbeater.cc:347] Connected to a master server at 127.0.99.62:32921
I20260619 21:14:38.028981 705 heartbeater.cc:464] Registering TS with master...
I20260619 21:14:38.030432 705 heartbeater.cc:511] Master 127.0.99.62:32921 requested a full tablet report, sending...
I20260619 21:14:38.033473 428 ts_manager.cc:194] Registered new tserver with Master: 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:38.033603 396 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.027413729s
I20260619 21:14:38.036890 428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38242
I20260619 21:14:38.041980 396 test_util.cc:268] Using random seed: -943657133
I20260619 21:14:38.130904 428 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38244:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 21:14:38.136709 428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 21:14:38.276512 556 heartbeater.cc:503] Master 127.0.99.62:32921 was elected leader, sending a full tablet report...
I20260619 21:14:38.362249 667 tablet_service.cc:1467] Processing CreateTablet for tablet d52855f5047c43ffacf998eec63c1b38 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260619 21:14:38.356142 669 tablet_service.cc:1467] Processing CreateTablet for tablet cd7a148bc32a405ba2dbf5c5e78eec41 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260619 21:14:38.361943 668 tablet_service.cc:1467] Processing CreateTablet for tablet 1cd5249877784e9596d4250a1d88d49e (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260619 21:14:38.364096 670 tablet_service.cc:1467] Processing CreateTablet for tablet aeb7af6b7887456382ec990dec4c06d6 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260619 21:14:38.368988 667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d52855f5047c43ffacf998eec63c1b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.369014 666 tablet_service.cc:1467] Processing CreateTablet for tablet 9deb38fccd1f42acaaec7a7b57b3fc5e (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260619 21:14:38.376955 665 tablet_service.cc:1467] Processing CreateTablet for tablet d82359fd768143449f4fe443d1879af6 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260619 21:14:38.377452 664 tablet_service.cc:1467] Processing CreateTablet for tablet b15841e7a6be4a1f84f7e64f03f6b0e8 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260619 21:14:38.381464 670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb7af6b7887456382ec990dec4c06d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.387713 663 tablet_service.cc:1467] Processing CreateTablet for tablet 8e14f261c84d4d6a8c9f42f4097f8ecb (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260619 21:14:38.391981 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15841e7a6be4a1f84f7e64f03f6b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.397200 662 tablet_service.cc:1467] Processing CreateTablet for tablet e48c29301ab04ff5bf0c2e96e3196ac1 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260619 21:14:38.402805 666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9deb38fccd1f42acaaec7a7b57b3fc5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.405536 662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48c29301ab04ff5bf0c2e96e3196ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.407634 663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e14f261c84d4d6a8c9f42f4097f8ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.416606 668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cd5249877784e9596d4250a1d88d49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.420674 665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82359fd768143449f4fe443d1879af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.429776 596 tablet_service.cc:1467] Processing CreateTablet for tablet aeb7af6b7887456382ec990dec4c06d6 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260619 21:14:38.433220 669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd7a148bc32a405ba2dbf5c5e78eec41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.444387 592 tablet_service.cc:1467] Processing CreateTablet for tablet 9deb38fccd1f42acaaec7a7b57b3fc5e (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260619 21:14:38.431963 595 tablet_service.cc:1467] Processing CreateTablet for tablet cd7a148bc32a405ba2dbf5c5e78eec41 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260619 21:14:38.433463 594 tablet_service.cc:1467] Processing CreateTablet for tablet 1cd5249877784e9596d4250a1d88d49e (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260619 21:14:38.451867 591 tablet_service.cc:1467] Processing CreateTablet for tablet d82359fd768143449f4fe443d1879af6 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260619 21:14:38.444519 593 tablet_service.cc:1467] Processing CreateTablet for tablet d52855f5047c43ffacf998eec63c1b38 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260619 21:14:38.458418 596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb7af6b7887456382ec990dec4c06d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.474803 518 tablet_service.cc:1467] Processing CreateTablet for tablet d52855f5047c43ffacf998eec63c1b38 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260619 21:14:38.476560 591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82359fd768143449f4fe443d1879af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.472607 519 tablet_service.cc:1467] Processing CreateTablet for tablet 1cd5249877784e9596d4250a1d88d49e (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260619 21:14:38.478891 590 tablet_service.cc:1467] Processing CreateTablet for tablet b15841e7a6be4a1f84f7e64f03f6b0e8 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260619 21:14:38.469262 521 tablet_service.cc:1467] Processing CreateTablet for tablet aeb7af6b7887456382ec990dec4c06d6 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260619 21:14:38.495651 595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd7a148bc32a405ba2dbf5c5e78eec41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.497963 517 tablet_service.cc:1467] Processing CreateTablet for tablet 9deb38fccd1f42acaaec7a7b57b3fc5e (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260619 21:14:38.501884 588 tablet_service.cc:1467] Processing CreateTablet for tablet e48c29301ab04ff5bf0c2e96e3196ac1 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260619 21:14:38.510258 594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cd5249877784e9596d4250a1d88d49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.489583 589 tablet_service.cc:1467] Processing CreateTablet for tablet 8e14f261c84d4d6a8c9f42f4097f8ecb (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260619 21:14:38.498795 520 tablet_service.cc:1467] Processing CreateTablet for tablet cd7a148bc32a405ba2dbf5c5e78eec41 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260619 21:14:38.513916 516 tablet_service.cc:1467] Processing CreateTablet for tablet d82359fd768143449f4fe443d1879af6 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260619 21:14:38.524788 518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d52855f5047c43ffacf998eec63c1b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.526182 515 tablet_service.cc:1467] Processing CreateTablet for tablet b15841e7a6be4a1f84f7e64f03f6b0e8 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260619 21:14:38.527714 519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cd5249877784e9596d4250a1d88d49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.530517 514 tablet_service.cc:1467] Processing CreateTablet for tablet 8e14f261c84d4d6a8c9f42f4097f8ecb (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260619 21:14:38.532725 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb7af6b7887456382ec990dec4c06d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.527982 590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15841e7a6be4a1f84f7e64f03f6b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.535719 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd7a148bc32a405ba2dbf5c5e78eec41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.541569 589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e14f261c84d4d6a8c9f42f4097f8ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.542702 516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82359fd768143449f4fe443d1879af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.547047 513 tablet_service.cc:1467] Processing CreateTablet for tablet e48c29301ab04ff5bf0c2e96e3196ac1 (DEFAULT_TABLE table=test-workload [id=c63044d086994eb4b77b8e0a1bda8ccc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260619 21:14:38.550199 514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e14f261c84d4d6a8c9f42f4097f8ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.551940 588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48c29301ab04ff5bf0c2e96e3196ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.556432 515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15841e7a6be4a1f84f7e64f03f6b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.561844 513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48c29301ab04ff5bf0c2e96e3196ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.610231 593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d52855f5047c43ffacf998eec63c1b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.601984 517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9deb38fccd1f42acaaec7a7b57b3fc5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.612908 592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9deb38fccd1f42acaaec7a7b57b3fc5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:14:38.667719 631 heartbeater.cc:503] Master 127.0.99.62:32921 was elected leader, sending a full tablet report...
W20260619 21:14:38.680176 476 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:14:38.758450 724 tablet_bootstrap.cc:492] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:38.846506 725 tablet_bootstrap.cc:492] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:38.852996 726 tablet_bootstrap.cc:492] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:38.862756 724 tablet_bootstrap.cc:654] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:38.878734 726 tablet_bootstrap.cc:654] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:38.892438 724 tablet_bootstrap.cc:492] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:38.893589 724 ts_tablet_manager.cc:1397] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.153s user 0.007s sys 0.025s
I20260619 21:14:38.901180 724 raft_consensus.cc:348] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.902000 725 tablet_bootstrap.cc:654] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:38.902724 726 tablet_bootstrap.cc:492] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:38.902738 724 raft_consensus.cc:374] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:38.903450 726 ts_tablet_manager.cc:1397] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.023s
I20260619 21:14:38.903340 724 raft_consensus.cc:729] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:38.905105 724 consensus_queue.cc:260] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.908654 726 raft_consensus.cc:348] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.909706 726 raft_consensus.cc:374] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:38.910122 726 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:38.911082 726 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.921072 724 ts_tablet_manager.cc:1428] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.027s user 0.015s sys 0.005s
I20260619 21:14:38.922294 725 tablet_bootstrap.cc:492] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:38.922842 725 ts_tablet_manager.cc:1397] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.086s user 0.011s sys 0.015s
I20260619 21:14:38.924707 726 ts_tablet_manager.cc:1428] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.021s user 0.009s sys 0.009s
I20260619 21:14:38.926162 726 tablet_bootstrap.cc:492] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:38.929563 705 heartbeater.cc:503] Master 127.0.99.62:32921 was elected leader, sending a full tablet report...
I20260619 21:14:38.938445 726 tablet_bootstrap.cc:654] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:38.928895 725 raft_consensus.cc:348] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.939700 725 raft_consensus.cc:374] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:38.940168 725 raft_consensus.cc:729] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:38.941306 725 consensus_queue.cc:260] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.945856 724 tablet_bootstrap.cc:492] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
W20260619 21:14:38.949822 557 tablet.cc:2369] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 21:14:38.964150 725 ts_tablet_manager.cc:1428] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.041s user 0.010s sys 0.020s
I20260619 21:14:38.965679 725 tablet_bootstrap.cc:492] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:38.970760 724 tablet_bootstrap.cc:654] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:38.974854 726 tablet_bootstrap.cc:492] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:38.977967 726 ts_tablet_manager.cc:1397] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.052s user 0.024s sys 0.018s
I20260619 21:14:38.979333 725 tablet_bootstrap.cc:654] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:38.984074 726 raft_consensus.cc:348] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.988004 726 raft_consensus.cc:374] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:38.988442 726 raft_consensus.cc:729] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:38.989668 726 consensus_queue.cc:260] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:38.992682 725 tablet_bootstrap.cc:492] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:38.993443 725 ts_tablet_manager.cc:1397] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.015s
I20260619 21:14:38.993641 724 tablet_bootstrap.cc:492] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:38.993803 726 ts_tablet_manager.cc:1428] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.015s user 0.002s sys 0.009s
I20260619 21:14:38.994333 724 ts_tablet_manager.cc:1397] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.049s user 0.012s sys 0.018s
I20260619 21:14:38.995549 726 tablet_bootstrap.cc:492] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:39.002111 725 raft_consensus.cc:348] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.003376 725 raft_consensus.cc:374] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.003818 725 raft_consensus.cc:729] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.004339 731 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.004910 725 consensus_queue.cc:260] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.005667 724 raft_consensus.cc:348] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.006731 724 raft_consensus.cc:374] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.007195 724 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.005055 731 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.011041 725 ts_tablet_manager.cc:1428] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.017s user 0.008s sys 0.004s
I20260619 21:14:39.012348 725 tablet_bootstrap.cc:492] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:39.015515 731 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.020452 731 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.022413 731 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.020701 724 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.030562 731 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.034078 726 tablet_bootstrap.cc:654] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.043062 724 ts_tablet_manager.cc:1428] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.048s user 0.008s sys 0.006s
I20260619 21:14:39.045641 724 tablet_bootstrap.cc:492] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:39.059906 725 tablet_bootstrap.cc:654] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.067448 724 tablet_bootstrap.cc:654] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.110668 726 tablet_bootstrap.cc:492] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:39.111572 726 ts_tablet_manager.cc:1397] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.116s user 0.022s sys 0.009s
I20260619 21:14:39.116499 725 tablet_bootstrap.cc:492] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:39.117405 725 ts_tablet_manager.cc:1397] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.105s user 0.008s sys 0.029s
I20260619 21:14:39.119844 726 raft_consensus.cc:348] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.122502 724 tablet_bootstrap.cc:492] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:39.123068 724 ts_tablet_manager.cc:1397] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.078s user 0.017s sys 0.010s
I20260619 21:14:39.121095 726 raft_consensus.cc:374] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.122702 725 raft_consensus.cc:348] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.123739 726 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.124305 725 raft_consensus.cc:374] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.124810 725 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.124831 726 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.126279 725 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.127144 724 raft_consensus.cc:348] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.128144 724 raft_consensus.cc:374] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.128495 724 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.129662 724 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.132584 732 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.133692 732 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.142406 732 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.140827 724 ts_tablet_manager.cc:1428] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.017s user 0.010s sys 0.003s
I20260619 21:14:39.144587 725 ts_tablet_manager.cc:1428] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.027s user 0.010s sys 0.001s
I20260619 21:14:39.145320 726 ts_tablet_manager.cc:1428] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.033s user 0.009s sys 0.002s
I20260619 21:14:39.148224 724 tablet_bootstrap.cc:492] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:39.151932 725 tablet_bootstrap.cc:492] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:39.153911 726 tablet_bootstrap.cc:492] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:39.172029 606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.171813 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.176568 605 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 0.
I20260619 21:14:39.177188 606 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 0.
I20260619 21:14:39.182433 495 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:39.184551 731 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.185110 731 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.184794 495 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:39.185653 731 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.192114 729 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.193027 729 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.204376 724 tablet_bootstrap.cc:654] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.216238 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.234351 729 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:39.235642 729 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.236359 729 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.245884 729 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:39.214769 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.271178 726 tablet_bootstrap.cc:654] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.287815 729 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.288769 729 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
W20260619 21:14:39.284399 494 leader_election.cc:341] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.298309 729 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:39.279263 725 tablet_bootstrap.cc:654] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.301962 746 raft_consensus.cc:2781] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.302656 746 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.303200 746 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:14:39.299062 494 leader_election.cc:341] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:14:39.278563 495 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:33541, user_credentials={real_user=slave}} blocked reactor thread for 93846.3us
I20260619 21:14:39.329456 724 tablet_bootstrap.cc:492] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:39.331346 724 ts_tablet_manager.cc:1397] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.184s user 0.028s sys 0.017s
I20260619 21:14:39.332494 731 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.344301 731 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.346235 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:39.347777 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:39.348089 605 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:14:39.355587 494 leader_election.cc:341] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.358894 724 raft_consensus.cc:348] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.360677 724 raft_consensus.cc:374] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.361095 724 raft_consensus.cc:729] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.363615 725 tablet_bootstrap.cc:492] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:39.362268 724 consensus_queue.cc:260] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.365010 732 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.369855 725 ts_tablet_manager.cc:1397] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.218s user 0.013s sys 0.040s
I20260619 21:14:39.368772 732 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.374219 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.376480 732 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
W20260619 21:14:39.378563 569 leader_election.cc:341] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.381662 747 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.382879 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.383134 724 ts_tablet_manager.cc:1428] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.045s user 0.007s sys 0.005s
I20260619 21:14:39.384256 725 raft_consensus.cc:348] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.382706 747 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.387847 725 raft_consensus.cc:374] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260619 21:14:39.386010 569 leader_election.cc:341] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.387818 726 tablet_bootstrap.cc:492] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:39.390066 726 ts_tablet_manager.cc:1397] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.236s user 0.024s sys 0.034s
I20260619 21:14:39.388230 724 tablet_bootstrap.cc:492] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:39.389079 725 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.393064 725 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.394147 747 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:39.398156 725 ts_tablet_manager.cc:1428] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.028s user 0.010s sys 0.002s
I20260619 21:14:39.402861 725 tablet_bootstrap.cc:492] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:39.405217 726 raft_consensus.cc:348] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.406262 726 raft_consensus.cc:374] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.406701 726 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.408427 726 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.416676 726 ts_tablet_manager.cc:1428] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.026s user 0.007s sys 0.008s
I20260619 21:14:39.418969 726 tablet_bootstrap.cc:492] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:39.430686 606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.436609 605 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 1.
I20260619 21:14:39.436360 746 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.444597 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.443097 606 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 0.
I20260619 21:14:39.445168 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.447149 495 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters: 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:39.443638 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.454823 644 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:39.455263 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.457909 747 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.458634 747 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.459259 747 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:14:39.462509 571 leader_election.cc:341] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.463271 571 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:39.465618 732 raft_consensus.cc:2726] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260619 21:14:39.464140 644 leader_election.cc:341] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.466888 725 tablet_bootstrap.cc:654] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
W20260619 21:14:39.467252 644 leader_election.cc:341] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.449043 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.468220 602 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 0.
I20260619 21:14:39.461688 531 raft_consensus.cc:2379] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 18e29b563e474386ac80d2afc78e3690 in current term 1: Already voted for candidate 86439c50df6044b7be9eccfcf6066c1a in this term.
I20260619 21:14:39.473524 731 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:39.473996 724 tablet_bootstrap.cc:654] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.474979 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:39.475839 679 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.469802 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:39.477025 746 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.484860 571 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:39.486123 732 raft_consensus.cc:2726] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:14:39.486972 603 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.487957 731 raft_consensus.cc:686] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:39.483579 644 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:39.500823 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.501322 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.502199 530 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 0.
I20260619 21:14:39.505342 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
W20260619 21:14:39.505194 645 leader_election.cc:341] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.505122 731 consensus_queue.cc:237] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.505549 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.520592 528 raft_consensus.cc:2379] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 000d8010fc7a4d3ca6fced1e06386c3f in current term 1: Already voted for candidate 86439c50df6044b7be9eccfcf6066c1a in this term.
I20260619 21:14:39.517514 645 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters: 18e29b563e474386ac80d2afc78e3690
W20260619 21:14:39.527047 645 leader_election.cc:341] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.527884 645 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f; no voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:39.530491 679 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 1.
I20260619 21:14:39.531584 747 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.533344 494 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:39.536679 729 raft_consensus.cc:2741] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260619 21:14:39.538094 754 raft_consensus.cc:2781] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:39.538900 754 raft_consensus.cc:686] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:39.541085 729 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.541579 729 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.540450 754 consensus_queue.cc:237] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.550161 729 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.558416 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:39.556552 747 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:39.560948 602 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.572144 746 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.561805 747 raft_consensus.cc:2726] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:14:39.572971 746 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.577867 606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.582473 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
W20260619 21:14:39.583459 495 leader_election.cc:341] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.579320 746 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
W20260619 21:14:39.584546 494 leader_election.cc:341] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.585129 494 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86439c50df6044b7be9eccfcf6066c1a; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:39.586689 726 tablet_bootstrap.cc:654] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.587095 731 raft_consensus.cc:2726] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260619 21:14:39.592954 645 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:40099, user_credentials={real_user=slave}} blocked reactor thread for 65949.6us
I20260619 21:14:39.595854 428 catalog_manager.cc:5495] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 1 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:39.597523 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:39.598394 724 tablet_bootstrap.cc:492] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:39.598475 725 tablet_bootstrap.cc:492] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:39.599086 724 ts_tablet_manager.cc:1397] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.213s user 0.017s sys 0.014s
W20260619 21:14:39.600517 645 leader_election.cc:341] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.606003 724 raft_consensus.cc:348] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:39.607154 724 raft_consensus.cc:374] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.607613 724 raft_consensus.cc:729] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.608659 724 consensus_queue.cc:260] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:39.612609 724 ts_tablet_manager.cc:1428] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.013s user 0.006s sys 0.002s
I20260619 21:14:39.613246 725 ts_tablet_manager.cc:1397] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.211s user 0.016s sys 0.043s
I20260619 21:14:39.619707 725 raft_consensus.cc:348] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.620963 725 raft_consensus.cc:374] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.621418 725 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.625000 724 tablet_bootstrap.cc:492] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:39.632570 725 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.637951 729 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.640637 729 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:39.642295 606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:39.655449 725 ts_tablet_manager.cc:1428] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.042s user 0.008s sys 0.004s
W20260619 21:14:39.655694 644 leader_election.cc:341] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.642652 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:39.656708 725 tablet_bootstrap.cc:492] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:39.656713 528 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.637707 603 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 1.
I20260619 21:14:39.670594 726 tablet_bootstrap.cc:492] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:39.671378 726 ts_tablet_manager.cc:1397] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.253s user 0.015s sys 0.037s
W20260619 21:14:39.685093 476 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:14:39.692878 726 raft_consensus.cc:348] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.694195 726 raft_consensus.cc:374] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.694659 726 raft_consensus.cc:729] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.695858 725 tablet_bootstrap.cc:654] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.697923 724 tablet_bootstrap.cc:654] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.704627 726 consensus_queue.cc:260] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.706312 754 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.707039 754 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.711440 726 ts_tablet_manager.cc:1428] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.040s user 0.007s sys 0.006s
I20260619 21:14:39.713228 726 tablet_bootstrap.cc:492] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:39.713704 754 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.714968 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.716276 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.717192 679 raft_consensus.cc:2454] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 0.
I20260619 21:14:39.718626 494 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
W20260619 21:14:39.719812 495 leader_election.cc:341] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.720258 754 raft_consensus.cc:2781] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.720811 754 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.721366 754 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.727279 724 tablet_bootstrap.cc:492] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:39.729118 726 tablet_bootstrap.cc:654] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.729542 725 tablet_bootstrap.cc:492] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:39.729854 724 ts_tablet_manager.cc:1397] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.105s user 0.011s sys 0.015s
I20260619 21:14:39.730204 725 ts_tablet_manager.cc:1397] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.074s user 0.012s sys 0.025s
I20260619 21:14:39.735097 602 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 1.
I20260619 21:14:39.735190 528 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 1.
I20260619 21:14:39.736831 644 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters: 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:39.737078 645 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters: 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:39.737807 725 raft_consensus.cc:348] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:39.738504 729 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:39.739473 759 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:39.740453 759 raft_consensus.cc:686] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:39.740128 729 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:39.749776 729 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.744310 759 consensus_queue.cc:237] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.738869 725 raft_consensus.cc:374] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.752786 725 raft_consensus.cc:729] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.751871 724 raft_consensus.cc:348] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.760609 724 raft_consensus.cc:374] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.762120 724 raft_consensus.cc:729] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.764989 724 consensus_queue.cc:260] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.769937 724 ts_tablet_manager.cc:1428] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.040s user 0.013s sys 0.002s
I20260619 21:14:39.771684 724 tablet_bootstrap.cc:492] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:39.764101 725 consensus_queue.cc:260] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:39.799983 725 ts_tablet_manager.cc:1428] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.069s user 0.010s sys 0.006s
I20260619 21:14:39.802486 725 tablet_bootstrap.cc:492] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:39.803310 726 tablet_bootstrap.cc:492] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:39.804062 726 ts_tablet_manager.cc:1397] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.091s user 0.011s sys 0.027s
I20260619 21:14:39.813499 725 tablet_bootstrap.cc:654] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.815048 726 raft_consensus.cc:348] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.818935 726 raft_consensus.cc:374] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.820276 726 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.824127 724 tablet_bootstrap.cc:654] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.823737 726 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.826861 731 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.827545 731 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.832216 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:39.833359 602 raft_consensus.cc:2379] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86439c50df6044b7be9eccfcf6066c1a in current term 1: Already voted for candidate 000d8010fc7a4d3ca6fced1e06386c3f in this term.
I20260619 21:14:39.834090 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.828419 428 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 1 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:39.831642 726 ts_tablet_manager.cc:1428] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.027s user 0.004s sys 0.007s
I20260619 21:14:39.837291 495 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86439c50df6044b7be9eccfcf6066c1a; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:39.841884 732 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.838294 726 tablet_bootstrap.cc:492] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:39.842536 732 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:39.843263 731 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.841336 754 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.843457 725 tablet_bootstrap.cc:492] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:39.844357 725 ts_tablet_manager.cc:1397] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.042s user 0.020s sys 0.014s
I20260619 21:14:39.845770 754 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.846415 746 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.849947 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.836009 428 catalog_manager.cc:5495] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 1 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:39.851769 732 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:39.855849 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.849956 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:39.857473 680 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 0.
W20260619 21:14:39.858506 495 leader_election.cc:341] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.859722 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:39.860708 679 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.864872 569 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:39.872493 732 raft_consensus.cc:2781] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.873064 732 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.873508 732 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.867955 725 raft_consensus.cc:348] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.877314 725 raft_consensus.cc:374] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.879977 725 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.881037 725 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
W20260619 21:14:39.891650 571 leader_election.cc:341] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.895330 725 ts_tablet_manager.cc:1428] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.051s user 0.001s sys 0.021s
I20260619 21:14:39.897122 724 tablet_bootstrap.cc:492] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:39.899818 724 ts_tablet_manager.cc:1397] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.128s user 0.013s sys 0.034s
I20260619 21:14:39.907877 773 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:39.907935 725 tablet_bootstrap.cc:492] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:39.908608 773 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.911067 724 raft_consensus.cc:348] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.912024 724 raft_consensus.cc:374] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:39.912423 724 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:39.913364 724 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.918169 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:39.919085 528 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 0.
I20260619 21:14:39.920605 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:39.921516 680 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 0.
I20260619 21:14:39.917634 773 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:39.923308 571 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:39.922224 426 catalog_manager.cc:5495] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 1 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:39.924603 773 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:39.925745 724 ts_tablet_manager.cc:1428] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.025s user 0.006s sys 0.005s
I20260619 21:14:39.925912 773 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:39.926313 773 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.927280 724 tablet_bootstrap.cc:492] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:39.931318 725 tablet_bootstrap.cc:654] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.944309 746 raft_consensus.cc:2726] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:14:39.948757 732 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:39.951383 726 tablet_bootstrap.cc:654] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.951871 732 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Requested vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:39.959796 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:39.953836 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:39.961745 680 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:39.953259 679 raft_consensus.cc:2454] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 1.
W20260619 21:14:39.962747 571 leader_election.cc:341] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:14:39.973093 746 consensus_queue.cc:1035] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:39.972226 494 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters: 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:39.977527 731 raft_consensus.cc:2781] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:39.978220 731 raft_consensus.cc:686] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
W20260619 21:14:39.983901 494 consensus_peers.cc:487] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a -> Peer 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523): Couldn't send request to peer 000d8010fc7a4d3ca6fced1e06386c3f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:14:39.979388 731 consensus_queue.cc:237] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:39.995971 724 tablet_bootstrap.cc:654] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:39.996469 725 tablet_bootstrap.cc:492] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:39.996833 726 tablet_bootstrap.cc:492] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:39.997105 725 ts_tablet_manager.cc:1397] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.090s user 0.018s sys 0.014s
I20260619 21:14:40.003221 726 ts_tablet_manager.cc:1397] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.165s user 0.008s sys 0.035s
I20260619 21:14:40.026257 725 raft_consensus.cc:348] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.027379 725 raft_consensus.cc:374] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.027709 725 raft_consensus.cc:729] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.028575 725 consensus_queue.cc:260] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.031442 725 ts_tablet_manager.cc:1428] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.033s user 0.008s sys 0.004s
I20260619 21:14:40.033154 725 tablet_bootstrap.cc:492] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: Bootstrap starting.
I20260619 21:14:40.041704 425 catalog_manager.cc:5495] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 1 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:40.047250 726 raft_consensus.cc:348] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.048204 726 raft_consensus.cc:374] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.048578 726 raft_consensus.cc:729] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.051133 724 tablet_bootstrap.cc:492] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:40.062314 724 ts_tablet_manager.cc:1397] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.135s user 0.021s sys 0.012s
I20260619 21:14:40.062031 754 consensus_queue.cc:1035] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.060716 726 consensus_queue.cc:260] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.066773 726 ts_tablet_manager.cc:1428] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.063s user 0.003s sys 0.008s
I20260619 21:14:40.068044 726 tablet_bootstrap.cc:492] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:40.051797 725 tablet_bootstrap.cc:654] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:40.053697 773 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.072870 773 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:40.055078 680 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 1.
I20260619 21:14:40.077396 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:40.082741 528 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.079349 724 raft_consensus.cc:348] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.084261 724 raft_consensus.cc:374] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.087352 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:40.083706 569 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters: 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:40.096181 724 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.097471 680 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.098109 773 raft_consensus.cc:2781] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:40.098941 773 raft_consensus.cc:686] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:40.100384 773 consensus_queue.cc:237] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:40.104184 724 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.114216 724 ts_tablet_manager.cc:1428] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.051s user 0.008s sys 0.001s
I20260619 21:14:40.115375 724 tablet_bootstrap.cc:492] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f: Bootstrap starting.
I20260619 21:14:40.118167 746 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.129256 726 tablet_bootstrap.cc:654] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:40.154194 528 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 1.
I20260619 21:14:40.156620 571 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:40.158030 773 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:40.158810 773 raft_consensus.cc:686] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:40.160300 773 consensus_queue.cc:237] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.176854 725 tablet_bootstrap.cc:492] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: No bootstrap required, opened a new log
I20260619 21:14:40.177395 725 ts_tablet_manager.cc:1397] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: Time spent bootstrapping tablet: real 0.144s user 0.010s sys 0.010s
I20260619 21:14:40.187968 725 raft_consensus.cc:348] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.189055 725 raft_consensus.cc:374] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.189415 725 raft_consensus.cc:729] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.190225 725 consensus_queue.cc:260] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.203202 724 tablet_bootstrap.cc:654] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:40.204818 725 ts_tablet_manager.cc:1428] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: Time spent starting tablet: real 0.027s user 0.008s sys 0.003s
I20260619 21:14:40.207504 726 tablet_bootstrap.cc:492] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:40.208060 726 ts_tablet_manager.cc:1397] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.140s user 0.023s sys 0.019s
I20260619 21:14:40.212586 726 raft_consensus.cc:348] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:40.213585 726 raft_consensus.cc:374] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.213883 726 raft_consensus.cc:729] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.214641 726 consensus_queue.cc:260] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:40.216778 726 ts_tablet_manager.cc:1428] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260619 21:14:40.217840 726 tablet_bootstrap.cc:492] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a: Bootstrap starting.
I20260619 21:14:40.233215 773 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:40.234743 726 tablet_bootstrap.cc:654] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a: Neither blocks nor log segments found. Creating new log.
I20260619 21:14:40.234048 773 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.236902 724 tablet_bootstrap.cc:492] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f: No bootstrap required, opened a new log
I20260619 21:14:40.237749 724 ts_tablet_manager.cc:1397] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent bootstrapping tablet: real 0.123s user 0.010s sys 0.011s
I20260619 21:14:40.240233 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:40.240588 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:40.241715 679 raft_consensus.cc:2379] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 18e29b563e474386ac80d2afc78e3690 in current term 1: Already voted for candidate 86439c50df6044b7be9eccfcf6066c1a in this term.
I20260619 21:14:40.248438 759 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.252298 680 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 1.
I20260619 21:14:40.252223 569 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:40.253713 774 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.254885 724 raft_consensus.cc:348] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.256286 724 raft_consensus.cc:374] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.256701 724 raft_consensus.cc:729] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.256350 426 catalog_manager.cc:5495] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 1 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:40.257763 724 consensus_queue.cc:260] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.241833 773 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:40.266343 724 ts_tablet_manager.cc:1428] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260619 21:14:40.268273 759 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:40.267689 426 catalog_manager.cc:5495] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 1 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:40.269011 759 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.274457 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:40.275453 529 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 0.
I20260619 21:14:40.278263 645 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:40.278894 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:40.279292 729 raft_consensus.cc:2781] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:40.279803 729 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:40.279942 603 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 0.
I20260619 21:14:40.280339 729 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.282079 759 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:40.293056 773 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:40.293871 773 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.297005 773 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:40.299001 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:40.300475 529 raft_consensus.cc:2379] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 18e29b563e474386ac80d2afc78e3690 in current term 1: Already voted for candidate 000d8010fc7a4d3ca6fced1e06386c3f in this term.
I20260619 21:14:40.302737 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:40.305138 569 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:40.306916 773 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.307687 759 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.312413 726 tablet_bootstrap.cc:492] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a: No bootstrap required, opened a new log
I20260619 21:14:40.312824 726 ts_tablet_manager.cc:1397] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a: Time spent bootstrapping tablet: real 0.095s user 0.017s sys 0.014s
I20260619 21:14:40.316012 726 raft_consensus.cc:348] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.316681 726 raft_consensus.cc:374] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:14:40.316967 726 raft_consensus.cc:729] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Initialized, Role: FOLLOWER
I20260619 21:14:40.317638 726 consensus_queue.cc:260] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.319765 726 ts_tablet_manager.cc:1428] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260619 21:14:40.351516 793 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.358289 774 raft_consensus.cc:2726] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:14:40.363840 729 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.366600 729 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:40.367743 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:40.368625 529 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.369355 774 raft_consensus.cc:482] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:40.370043 774 raft_consensus.cc:504] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.373538 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:40.374033 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:40.374418 603 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.374791 530 raft_consensus.cc:2454] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 0.
I20260619 21:14:40.376475 571 leader_election.cc:302] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:40.377830 732 raft_consensus.cc:2781] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:14:40.378480 732 raft_consensus.cc:482] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:40.378965 732 raft_consensus.cc:3037] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.385514 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:40.386562 680 raft_consensus.cc:2454] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 0.
I20260619 21:14:40.388743 774 leader_election.cc:288] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:40.464428 529 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 1.
I20260619 21:14:40.464552 732 raft_consensus.cc:504] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.466463 645 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:40.468269 729 raft_consensus.cc:2781] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:40.468904 732 leader_election.cc:288] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:40.468915 729 raft_consensus.cc:686] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:40.470549 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:40.471455 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:40.471756 680 raft_consensus.cc:3037] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.470603 729 consensus_queue.cc:237] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.473223 773 raft_consensus.cc:2726] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:14:40.472463 529 raft_consensus.cc:3037] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.484313 679 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.485271 793 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.504413 428 catalog_manager.cc:5495] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 1 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:40.506800 746 consensus_queue.cc:1035] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:14:40.507308 773 consensus_queue.cc:1035] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.530903 754 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:40.531551 754 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:40.534317 754 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:40.539882 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:40.543053 754 consensus_queue.cc:1035] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.545246 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:40.549278 528 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:14:40.551134 495 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86439c50df6044b7be9eccfcf6066c1a; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:40.560604 603 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 1.
I20260619 21:14:40.560592 680 raft_consensus.cc:2454] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 1.
I20260619 21:14:40.560592 529 raft_consensus.cc:2454] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 1.
I20260619 21:14:40.575096 571 leader_election.cc:302] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:40.576511 732 raft_consensus.cc:2781] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:14:40.577258 732 raft_consensus.cc:686] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:40.578572 732 consensus_queue.cc:237] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.610585 425 catalog_manager.cc:5495] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 1 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:40.668399 754 raft_consensus.cc:2726] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:14:40.674679 732 consensus_queue.cc:1035] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:40.679648 729 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:40.680325 729 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:40.688567 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:40.696465 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:40.698532 644 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f; no voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:40.700040 793 raft_consensus.cc:2726] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:14:40.699863 729 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:40.751490 732 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:14:40.769550 774 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:14:40.988555 773 consensus_queue.cc:1035] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:14:41.000138 808 consensus_queue.cc:1035] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:14:41.011554 729 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:14:41.054759 811 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:14:41.118678 602 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:41980
I20260619 21:14:41.119823 602 raft_consensus.cc:595] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Received request to transfer leadership to TS 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:41.130615 602 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:41980
I20260619 21:14:41.131423 602 raft_consensus.cc:595] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Received request to transfer leadership to TS 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:41.158901 680 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:57374
I20260619 21:14:41.159904 680 raft_consensus.cc:595] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:41.167476 773 raft_consensus.cc:979] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:41.167778 680 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d82359fd768143449f4fe443d1879af6"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:57374
I20260619 21:14:41.168143 773 raft_consensus.cc:1067] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Signalling peer 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:41.168623 680 raft_consensus.cc:595] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:41.170763 528 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:36814
I20260619 21:14:41.171576 528 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.172752 528 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.191052 529 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:36834
I20260619 21:14:41.191569 529 raft_consensus.cc:595] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:41.196825 680 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:57374
I20260619 21:14:41.197536 680 raft_consensus.cc:595] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:41.203135 602 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:41980
I20260619 21:14:41.203830 602 raft_consensus.cc:595] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Received request to transfer leadership to TS 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:41.221207 528 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.225294 528 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 2 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:41.225965 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:41.226432 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:41.226698 602 raft_consensus.cc:3032] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.227051 680 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.227027 602 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.228147 602 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.229673 602 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.307552 602 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 2.
I20260619 21:14:41.307552 680 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 2.
I20260619 21:14:41.309254 494 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:41.310300 754 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.310998 754 raft_consensus.cc:686] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 2 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:41.312309 754 consensus_queue.cc:237] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.320482 428 catalog_manager.cc:5495] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 1 to 2, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 2 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:41.401525 731 raft_consensus.cc:979] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.402058 754 raft_consensus.cc:1067] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.403618 602 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41960
I20260619 21:14:41.404150 602 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.404505 602 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.465233 811 raft_consensus.cc:979] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.465691 793 raft_consensus.cc:1067] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.467325 603 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82359fd768143449f4fe443d1879af6"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:41.467844 603 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.468146 603 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.479233 602 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.481587 602 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:41.482331 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:41.482919 528 raft_consensus.cc:3032] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.483217 528 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:41.483732 680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:41.484342 680 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.483958 528 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.485285 528 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.485385 603 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.488257 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:41.488474 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:41.488844 529 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.489248 679 raft_consensus.cc:3032] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.489588 679 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:41.490363 679 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.491447 679 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.491555 603 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:41.529552 773 raft_consensus.cc:979] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:41.529980 732 raft_consensus.cc:1067] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Signalling peer 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:41.531337 530 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:36814
I20260619 21:14:41.531817 530 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.532083 530 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.538182 811 raft_consensus.cc:979] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.538650 793 raft_consensus.cc:1067] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.539882 603 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:41.540314 603 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.540551 603 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.557154 808 raft_consensus.cc:979] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:41.557624 808 raft_consensus.cc:1067] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Signalling peer 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:41.559357 677 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:57364
I20260619 21:14:41.559967 677 raft_consensus.cc:482] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.560294 677 raft_consensus.cc:3037] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.575434 528 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.575460 680 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.576923 571 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:41.578028 732 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.578606 732 raft_consensus.cc:686] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.580330 732 consensus_queue.cc:237] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.582437 677 raft_consensus.cc:504] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.575481 603 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.585798 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:41.586450 528 raft_consensus.cc:3037] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.587678 677 leader_election.cc:288] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:41.590987 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:41.592173 602 raft_consensus.cc:3032] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.593881 602 raft_consensus.cc:729] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.593955 428 catalog_manager.cc:5495] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 1 to 2, leader changed from 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 2 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:41.594779 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:41.595121 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:41.595360 531 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.595696 677 raft_consensus.cc:3032] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.596007 677 raft_consensus.cc:729] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:41.596504 602 consensus_queue.cc:260] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.593554 603 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:41.597942 602 raft_consensus.cc:3037] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.596810 677 consensus_queue.cc:260] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.599303 677 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.638178 811 raft_consensus.cc:979] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.638660 793 raft_consensus.cc:1067] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:41.639988 603 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:41.640442 603 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:41.640723 603 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.659557 529 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.660000 602 raft_consensus.cc:2454] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:41.661535 571 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:41.662411 732 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.662858 732 raft_consensus.cc:686] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.663666 732 consensus_queue.cc:237] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.666632 644 leader_election.cc:302] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:41.667518 793 raft_consensus.cc:2781] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.667991 793 raft_consensus.cc:686] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:41.668820 793 consensus_queue.cc:237] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.659557 679 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.677583 425 catalog_manager.cc:5495] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 1 to 2, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 2 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:41.677538 428 catalog_manager.cc:5495] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 1 to 2, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 2 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:41.733572 679 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:41.734798 754 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:41.742698 602 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:41.744558 731 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:41.764014 531 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.763679 530 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:41.763697 603 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.764986 528 raft_consensus.cc:2454] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:41.765877 530 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 2 election: Requested vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:41.766096 603 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:41.763649 677 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.767238 571 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:41.767859 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:41.768447 602 raft_consensus.cc:3032] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.768560 732 raft_consensus.cc:2781] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.768867 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:41.769272 732 raft_consensus.cc:686] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.768786 602 raft_consensus.cc:729] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.769560 530 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.770211 602 consensus_queue.cc:260] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:41.770226 732 consensus_queue.cc:237] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.771610 602 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.772771 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:41.773519 679 raft_consensus.cc:3032] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:41.773852 679 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:41.772047 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:41.774643 677 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.777196 679 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.778493 679 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:41.779935 425 catalog_manager.cc:5495] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 1 to 2, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 2 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:41.845172 530 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.845171 677 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 2.
I20260619 21:14:41.845172 602 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 2.
I20260619 21:14:41.846647 571 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:41.845510 679 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:41.847185 494 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:41.847689 732 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.848691 754 raft_consensus.cc:2781] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:41.848976 732 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:41.849272 754 raft_consensus.cc:686] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:41.849882 732 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:41.850363 754 consensus_queue.cc:237] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:41.858693 425 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 1 to 2, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 2 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:41.860512 428 catalog_manager.cc:5495] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 1 to 2, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 2 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:42.019804 679 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.021083 732 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.029672 530 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.031332 773 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.170359 530 raft_consensus.cc:1261] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.171821 773 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.180837 679 raft_consensus.cc:1261] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.182225 830 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.187799 677 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.188982 830 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.197451 530 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.198666 830 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.244683 602 raft_consensus.cc:1261] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.246377 793 consensus_queue.cc:1035] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.255837 530 raft_consensus.cc:1261] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.264034 677 raft_consensus.cc:1261] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.266618 754 consensus_queue.cc:1035] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:14:42.267655 839 consensus_queue.cc:1035] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.295372 602 raft_consensus.cc:1261] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.296885 754 consensus_queue.cc:1035] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:42.335820 530 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.337055 830 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:14:42.349265 677 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:42.352308 830 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:44.231832 396 auto_leader_rebalancer-test.cc:269] The leader distribution is
I20260619 21:14:44.232654 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:44.253060 602 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:44.253669 602 raft_consensus.cc:595] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Received request to transfer leadership to TS 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:44.264849 530 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
mode: GRACEFUL
new_leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:36842
I20260619 21:14:44.265393 530 raft_consensus.cc:595] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Received request to transfer leadership to TS 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:44.266338 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260619 21:14:44.266809 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:44.538944 808 raft_consensus.cc:979] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:44.539506 854 raft_consensus.cc:1067] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Signalling peer 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:44.541462 677 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:57364
I20260619 21:14:44.542119 677 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:44.542510 677 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:44.548807 677 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:44.551065 677 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:44.552198 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:44.552599 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:44.552837 530 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:44.553387 602 raft_consensus.cc:3032] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:44.553730 602 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:44.554728 602 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:44.555907 602 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:44.560437 530 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:44.562016 645 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:44.562901 849 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:14:44.563621 602 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:44.565063 849 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:44.566252 849 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:44.574141 425 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 2 to 3, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 3 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:44.595383 862 raft_consensus.cc:979] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: : Instructing follower 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:44.596000 861 raft_consensus.cc:1067] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Signalling peer 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:44.598203 677 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:57358
I20260619 21:14:44.598881 677 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:44.599400 677 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:44.607149 677 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:44.609061 677 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:44.610198 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:44.610355 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:44.611102 530 raft_consensus.cc:3032] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:44.611176 602 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:44.611435 530 raft_consensus.cc:729] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:44.612326 530 consensus_queue.cc:260] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:44.613586 530 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:44.617561 602 raft_consensus.cc:2454] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:44.618814 644 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:44.619925 849 raft_consensus.cc:2781] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:44.620435 849 raft_consensus.cc:686] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:44.620540 530 raft_consensus.cc:2454] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:44.621577 849 consensus_queue.cc:237] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:44.629995 425 catalog_manager.cc:5495] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 1 to 2, leader changed from 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 2 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:44.977843 602 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:44.979959 849 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:14:44.992077 530 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:44.993673 849 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:45.064380 530 raft_consensus.cc:1261] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:45.065492 849 consensus_queue.cc:1035] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:14:45.074846 602 raft_consensus.cc:1261] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:45.076478 855 consensus_queue.cc:1035] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:14:45.268098 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:45.273921 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:45.274341 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:47.213027 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:47.217710 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:47.218087 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260619 21:14:47.457973 701 debug-util.cc:398] Leaking SignalData structure 0x7b080009d0a0 after lost signal to thread 399
W20260619 21:14:47.458884 701 debug-util.cc:398] Leaking SignalData structure 0x7b0800099fc0 after lost signal to thread 462
W20260619 21:14:47.459760 701 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5640 after lost signal to thread 555
W20260619 21:14:47.460582 701 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3ac0 after lost signal to thread 630
W20260619 21:14:47.461339 701 debug-util.cc:398] Leaking SignalData structure 0x7b080003df40 after lost signal to thread 704
I20260619 21:14:48.219120 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:48.227517 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:48.228060 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:49.229084 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:49.237278 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:49.237812 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:50.859905 887 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.867936 893 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.868691 893 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.886221 890 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.887199 890 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.860597 887 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.897710 887 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:50.891786 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:50.899293 531 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:50.906484 898 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.927361 893 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:50.927254 645 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:50.928197 898 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.928948 894 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:14:50.929567 894 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.930007 894 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:50.931152 898 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:50.932636 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:50.933182 896 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:50.933475 531 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:50.933794 896 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:50.935269 645 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:50.936311 896 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:50.936336 888 raft_consensus.cc:2781] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:14:50.937067 888 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.937461 888 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:50.951476 894 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.953953 890 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:50.954880 896 raft_consensus.cc:482] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 000d8010fc7a4d3ca6fced1e06386c3f)
I20260619 21:14:50.955963 892 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 000d8010fc7a4d3ca6fced1e06386c3f)
I20260619 21:14:50.958962 892 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.961484 892 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:50.962491 896 raft_consensus.cc:504] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.964879 896 leader_election.cc:288] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:50.975716 896 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:50.976411 896 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:50.978964 896 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:50.982633 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:50.983464 531 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:50.994701 531 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:50.995915 890 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:50.999954 890 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.003793 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.004251 890 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:51.004645 530 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 2.
I20260619 21:14:50.997702 894 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:51.006611 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:51.006467 645 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:51.007470 894 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:14:51.007807 894 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 18e29b563e474386ac80d2afc78e3690)
I20260619 21:14:51.007678 645 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:51.008212 894 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.008653 890 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:14:51.009824 890 raft_consensus.cc:686] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:51.010829 890 consensus_queue.cc:237] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.014919 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:51.015478 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:51.045502 890 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:51.046430 890 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:51.060150 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.065348 890 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:51.071723 894 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.090391 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:51.091233 531 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.006459 896 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 000d8010fc7a4d3ca6fced1e06386c3f)
I20260619 21:14:51.099274 896 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.108122 894 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:51.104713 531 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:51.114370 645 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:51.115626 894 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:14:51.117172 894 raft_consensus.cc:686] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:51.119781 894 consensus_queue.cc:237] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.132589 890 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:51.132682 893 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:51.133242 890 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.135625 890 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:51.137041 530 raft_consensus.cc:1226] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Rejecting Update request from peer 18e29b563e474386ac80d2afc78e3690 for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.133272 893 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.141935 893 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:51.142208 530 raft_consensus.cc:1226] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Rejecting Update request from peer 18e29b563e474386ac80d2afc78e3690 for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.140518 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.145823 896 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:51.146237 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.147873 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.150404 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.151230 679 raft_consensus.cc:1226] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Rejecting Update request from peer 18e29b563e474386ac80d2afc78e3690 for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.147630 888 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.152443 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.153479 680 raft_consensus.cc:1226] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Rejecting Update request from peer 18e29b563e474386ac80d2afc78e3690 for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.154026 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.154460 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.155030 677 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:51.155084 896 consensus_queue.cc:1046] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.155112 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.156198 892 raft_consensus.cc:3032] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:51.156586 892 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:51.156736 896 consensus_queue.cc:1046] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.156857 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48c29301ab04ff5bf0c2e96e3196ac1" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.157867 896 raft_consensus.cc:3032] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:51.157469 892 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.158237 896 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:51.158666 892 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.159032 896 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.159472 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:51.159966 896 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.161209 679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.162031 895 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:51.162606 677 raft_consensus.cc:1226] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Rejecting Update request from peer 18e29b563e474386ac80d2afc78e3690 for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.164951 679 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:51.165028 895 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.167035 895 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:51.167873 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.167882 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.163717 425 catalog_manager.cc:5495] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 2 to 3, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 3 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:51.169037 676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.172756 675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.173182 569 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters: 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.180734 569 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.181572 895 raft_consensus.cc:2781] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:14:51.181927 895 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:51.182039 896 raft_consensus.cc:2726] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:14:51.181969 569 leader_election.cc:302] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.182436 895 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.182468 675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:51.182869 896 raft_consensus.cc:2726] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:14:51.183429 896 consensus_queue.cc:1046] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.169711 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:51.184413 896 raft_consensus.cc:3032] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:51.184741 896 raft_consensus.cc:729] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:51.183568 569 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters: 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.162909 888 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:51.185997 892 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:14:51.186460 892 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 86439c50df6044b7be9eccfcf6066c1a)
I20260619 21:14:51.187011 892 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.189220 569 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.190369 569 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.191560 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.191977 606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.192878 606 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 3.
I20260619 21:14:51.192243 604 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 3.
I20260619 21:14:51.191210 896 consensus_queue.cc:260] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.196724 895 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:51.198328 602 raft_consensus.cc:1226] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Rejecting Update request from peer 86439c50df6044b7be9eccfcf6066c1a for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.199571 895 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 election: Requested vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:51.199877 892 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.200291 895 raft_consensus.cc:2726] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:14:51.201102 603 raft_consensus.cc:1226] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Rejecting Update request from peer 86439c50df6044b7be9eccfcf6066c1a for earlier term 2. Current term is 3. Ops: []
I20260619 21:14:51.201821 892 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:51.202342 892 raft_consensus.cc:2726] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:14:51.201054 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:51.202611 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:51.197556 896 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.205631 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.207424 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.208156 602 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:51.209833 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:51.213920 603 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:51.215746 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.216818 604 raft_consensus.cc:2379] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 000d8010fc7a4d3ca6fced1e06386c3f in current term 3: Already voted for candidate 18e29b563e474386ac80d2afc78e3690 in this term.
I20260619 21:14:51.217814 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.218468 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:51.218636 605 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:51.221592 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.222486 675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:51.222991 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:51.223644 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:51.224086 569 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.224694 604 raft_consensus.cc:2379] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 000d8010fc7a4d3ca6fced1e06386c3f in current term 3: Already voted for candidate 18e29b563e474386ac80d2afc78e3690 in this term.
I20260619 21:14:51.225348 569 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.226109 895 raft_consensus.cc:2726] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:14:51.226253 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:51.226703 895 raft_consensus.cc:2726] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:14:51.228700 602 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:51.232736 495 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters: 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:51.234229 495 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters: 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:51.234318 891 raft_consensus.cc:2741] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260619 21:14:51.235128 891 raft_consensus.cc:2741] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260619 21:14:51.237111 898 consensus_queue.cc:1046] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.237164 891 consensus_queue.cc:1046] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:14:51.237965 604 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:51.238195 898 raft_consensus.cc:3032] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:51.238344 891 raft_consensus.cc:3032] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:51.238608 898 raft_consensus.cc:729] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:51.238740 891 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:51.238595 644 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f; no voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.239753 898 consensus_queue.cc:260] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:51.239807 891 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.241099 888 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.241508 891 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.242455 644 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f; no voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.243521 894 raft_consensus.cc:2726] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:14:51.244135 898 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.244836 644 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f; no voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.245720 894 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.250574 644 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters: 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:51.251379 888 raft_consensus.cc:2726] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:14:51.251724 893 raft_consensus.cc:2781] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:14:51.252313 893 raft_consensus.cc:686] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:51.253481 893 consensus_queue.cc:237] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:51.254398 894 raft_consensus.cc:2726] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:14:51.266358 425 catalog_manager.cc:5495] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 2 to 3, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 3 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:51.267510 425 catalog_manager.cc:5495] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 2 to 3, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 3 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:51.527513 531 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:51.528688 893 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.533658 604 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:51.535037 893 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.580436 531 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:51.581697 894 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.591121 604 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:51.592659 890 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:14:51.819813 604 raft_consensus.cc:1261] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:51.821072 890 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:51.829200 531 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:51.837301 531 raft_consensus.cc:1261] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:51.838392 890 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:52.016402 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:52.038910 678 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:40594
I20260619 21:14:52.039731 678 raft_consensus.cc:595] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:52.042594 678 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:40594
I20260619 21:14:52.043193 678 raft_consensus.cc:595] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:52.045528 678 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d82359fd768143449f4fe443d1879af6"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:40594
I20260619 21:14:52.046030 678 raft_consensus.cc:595] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:52.047118 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260619 21:14:52.047501 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260619 21:14:52.145349 701 debug-util.cc:398] Leaking SignalData structure 0x7b080008f2c0 after lost signal to thread 399
W20260619 21:14:52.146346 701 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3860 after lost signal to thread 462
W20260619 21:14:52.147387 701 debug-util.cc:398] Leaking SignalData structure 0x7b080002cee0 after lost signal to thread 555
W20260619 21:14:52.148401 701 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8940 after lost signal to thread 630
W20260619 21:14:52.149367 701 debug-util.cc:398] Leaking SignalData structure 0x7b08001ed120 after lost signal to thread 704
I20260619 21:14:52.156167 894 raft_consensus.cc:979] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:52.156803 888 raft_consensus.cc:1067] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:52.159147 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82359fd768143449f4fe443d1879af6"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:52.159991 604 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:52.160545 604 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.167069 604 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.169023 604 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:52.170195 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:52.170390 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:52.170892 528 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.171175 678 raft_consensus.cc:3032] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:14:52.171651 678 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:52.172780 678 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.174449 678 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.177821 528 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.179464 571 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:52.180393 895 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:14:52.182489 895 raft_consensus.cc:686] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:52.183635 895 consensus_queue.cc:237] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.185154 678 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.191459 425 catalog_manager.cc:5495] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 3 to 4, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 4 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:52.298060 890 raft_consensus.cc:979] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:52.298568 894 raft_consensus.cc:1067] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:52.300184 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:52.300783 604 raft_consensus.cc:482] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:52.301169 604 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.307157 604 raft_consensus.cc:504] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.309247 604 leader_election.cc:288] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:52.310387 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:52.310717 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e14f261c84d4d6a8c9f42f4097f8ecb" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:52.311213 528 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.311457 678 raft_consensus.cc:3032] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:14:52.311955 678 raft_consensus.cc:729] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:52.313151 678 consensus_queue.cc:260] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.314851 678 raft_consensus.cc:3037] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.320719 528 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.324199 678 raft_consensus.cc:2454] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.325482 571 leader_election.cc:302] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:52.326995 895 raft_consensus.cc:2781] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:14:52.327623 895 raft_consensus.cc:686] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:52.328765 895 consensus_queue.cc:237] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.340523 425 catalog_manager.cc:5495] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 3 to 4, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 4 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:52.362609 890 raft_consensus.cc:979] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:52.362993 894 raft_consensus.cc:1067] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:52.364174 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:52.364578 604 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:52.364835 604 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.369709 604 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.371239 604 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:52.372009 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:52.372308 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:52.372540 528 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.372882 678 raft_consensus.cc:3032] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:14:52.373229 678 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:52.373917 678 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.374850 678 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.377182 528 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.378057 571 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:52.378703 895 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:14:52.379153 895 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:52.379897 678 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.380070 895 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.386785 425 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 3 to 4, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 4 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:52.801287 919 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:52.802006 919 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:52.804734 919 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:52.854678 898 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:52.855249 898 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.857679 898 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:52.863119 895 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:52.863570 895 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:52.865757 895 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:52.890465 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:52.891430 678 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 3.
I20260619 21:14:52.893605 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:52.894310 494 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:52.895527 530 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 3.
I20260619 21:14:52.894989 529 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:52.894724 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:52.896860 675 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:52.897046 678 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 3.
I20260619 21:14:52.897089 898 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:14:52.898810 571 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:52.598203 528 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:52.900437 919 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:14:52.900188 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:52.900897 919 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:52.901468 919 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.901588 678 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 3.
I20260619 21:14:52.902989 895 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:14:52.904443 895 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:14:52.914373 924 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260619 21:14:52.923882 569 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:52.925143 926 raft_consensus.cc:2781] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:14:52.925657 926 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:52.926158 926 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.935801 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:52.936455 926 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:52.940200 925 raft_consensus.cc:482] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:14:52.941314 925 raft_consensus.cc:504] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:52.945573 925 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:52.946210 926 leader_election.cc:288] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Requested vote from peers 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523), 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099)
I20260619 21:14:52.947186 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690" is_pre_election: true
I20260619 21:14:52.948365 603 raft_consensus.cc:2379] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86439c50df6044b7be9eccfcf6066c1a in current term 4: Already voted for candidate 18e29b563e474386ac80d2afc78e3690 in this term.
I20260619 21:14:52.949441 528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:52.950240 528 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.955050 530 raft_consensus.cc:1261] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:52.959549 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a" is_pre_election: true
I20260619 21:14:52.961644 926 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:14:52.965508 528 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:52.969161 898 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:14:52.971892 675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" is_pre_election: true
I20260619 21:14:52.969645 898 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.973857 675 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 3.
I20260619 21:14:52.975299 571 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:52.976974 926 raft_consensus.cc:2781] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:14:52.977243 676 raft_consensus.cc:1261] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:52.977496 675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb7af6b7887456382ec990dec4c06d6" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:52.977663 926 raft_consensus.cc:686] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:52.978392 675 raft_consensus.cc:3037] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:52.979573 926 consensus_queue.cc:237] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } }
I20260619 21:14:52.981173 494 leader_election.cc:302] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters: 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:52.991063 925 raft_consensus.cc:2741] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260619 21:14:52.992357 924 consensus_queue.cc:1035] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:14:53.020479 898 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:53.027164 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:53.029728 898 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 4 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:53.033412 678 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:53.041193 919 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:53.055065 676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:53.055886 676 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:53.056989 675 raft_consensus.cc:2454] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:53.059175 924 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260619 21:14:53.070101 602 raft_consensus.cc:2379] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86439c50df6044b7be9eccfcf6066c1a in current term 4: Already voted for candidate 18e29b563e474386ac80d2afc78e3690 in this term.
I20260619 21:14:53.080039 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:53.080700 919 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:53.081327 530 raft_consensus.cc:2379] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 18e29b563e474386ac80d2afc78e3690 in current term 4: Already voted for candidate 86439c50df6044b7be9eccfcf6066c1a in this term.
I20260619 21:14:53.091996 676 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 4.
I20260619 21:14:53.105297 494 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters: 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:53.106434 925 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:14:53.107148 925 raft_consensus.cc:686] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:53.108680 925 consensus_queue.cc:237] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:53.127784 428 catalog_manager.cc:5495] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:53.108770 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:53.129981 677 raft_consensus.cc:2379] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 18e29b563e474386ac80d2afc78e3690 in current term 4: Already voted for candidate 86439c50df6044b7be9eccfcf6066c1a in this term.
I20260619 21:14:53.134193 569 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690; no voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:53.135907 895 raft_consensus.cc:2726] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:14:53.156730 425 catalog_manager.cc:5495] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 2 to 4, leader changed from 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 4 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:53.305404 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:53.312039 602 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:53.312600 602 raft_consensus.cc:595] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Received request to transfer leadership to TS 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:53.314023 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260619 21:14:53.314421 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:53.523375 677 raft_consensus.cc:1261] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:14:53.524624 895 consensus_queue.cc:1035] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:53.533476 528 raft_consensus.cc:1261] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:14:53.535808 895 consensus_queue.cc:1035] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:14:53.556119 927 raft_consensus.cc:979] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:53.556587 895 raft_consensus.cc:1067] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Signalling peer 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:53.558108 528 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:36814
I20260619 21:14:53.558663 528 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:53.559051 528 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:53.565567 528 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:53.567406 528 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 5 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:53.568331 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:53.568507 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:53.568874 602 raft_consensus.cc:3032] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:14:53.569226 602 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:53.569233 677 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:53.569862 602 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:53.570750 602 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:53.574352 677 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 5.
I20260619 21:14:53.575039 602 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 5.
I20260619 21:14:53.575544 494 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:53.576364 925 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:14:53.577023 925 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:53.577924 925 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:53.583921 428 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 4 to 5, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 5 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:53.594017 677 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:14:53.595211 925 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:53.606454 602 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:14:53.608285 925 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260619 21:14:53.915835 627 debug-util.cc:398] Leaking SignalData structure 0x7b080012a060 after lost signal to thread 399
W20260619 21:14:53.916779 627 debug-util.cc:398] Leaking SignalData structure 0x7b080012a740 after lost signal to thread 462
W20260619 21:14:53.917780 627 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3040 after lost signal to thread 555
W20260619 21:14:53.918705 627 debug-util.cc:398] Leaking SignalData structure 0x7b0800139340 after lost signal to thread 630
W20260619 21:14:53.919644 627 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3220 after lost signal to thread 704
I20260619 21:14:53.964365 602 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260619 21:14:53.965734 925 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260619 21:14:53.975449 677 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260619 21:14:53.976977 925 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260619 21:14:54.315270 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:54.321871 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:54.322523 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:55.323707 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:55.329607 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:14:55.330019 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:56.357204 396 auto_leader_rebalancer-test.cc:288] The leader distribution is
I20260619 21:14:56.364526 530 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:36834
I20260619 21:14:56.365409 530 raft_consensus.cc:595] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:56.369910 530 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:36834
I20260619 21:14:56.370352 530 raft_consensus.cc:595] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:56.375088 677 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:57374
I20260619 21:14:56.375614 677 raft_consensus.cc:595] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:56.385987 530 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:36834
I20260619 21:14:56.386452 530 raft_consensus.cc:595] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:56.393543 677 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
mode: GRACEFUL
new_leader_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:57374
I20260619 21:14:56.394001 677 raft_consensus.cc:595] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Received request to transfer leadership to TS 18e29b563e474386ac80d2afc78e3690
I20260619 21:14:56.568732 956 raft_consensus.cc:979] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.569398 957 raft_consensus.cc:1067] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.571058 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41960
I20260619 21:14:56.571569 604 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:56.571933 604 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:56.578948 604 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.581848 604 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:56.582477 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:56.582979 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:56.583402 530 raft_consensus.cc:3032] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:14:56.583685 677 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:56.583807 530 raft_consensus.cc:729] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:56.584699 530 consensus_queue.cc:260] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.586290 530 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:14:56.588402 677 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:56.589546 569 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:56.591135 953 raft_consensus.cc:2781] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:14:56.593703 530 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 2.
I20260619 21:14:56.595098 953 raft_consensus.cc:686] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:56.596113 953 consensus_queue.cc:237] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.608692 428 catalog_manager.cc:5495] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 1 to 2, leader changed from 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 2 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:56.613469 957 raft_consensus.cc:979] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.613929 957 raft_consensus.cc:1067] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.615473 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41960
I20260619 21:14:56.616016 604 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:56.616467 604 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:14:56.623117 604 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.626036 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:56.626224 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:56.626778 530 raft_consensus.cc:3032] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:14:56.626791 677 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:14:56.627329 530 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:56.627259 604 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 6 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:56.629336 530 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.630759 530 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:14:56.632717 677 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 6.
I20260619 21:14:56.633697 569 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:56.634452 953 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Leader election won for term 6
I20260619 21:14:56.634867 953 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:56.635691 953 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.637718 530 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 6.
I20260619 21:14:56.643739 428 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 5 to 6, leader changed from 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 6 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:56.683944 956 raft_consensus.cc:979] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.684442 957 raft_consensus.cc:1067] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.685913 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41960
I20260619 21:14:56.686395 604 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:56.686697 604 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:56.691721 604 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.693182 604 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 5 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:56.694027 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:56.694216 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:56.694692 530 raft_consensus.cc:3032] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:14:56.694852 677 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:56.695061 530 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:56.695883 530 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.697194 530 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:56.699467 677 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 5.
I20260619 21:14:56.700299 569 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 18e29b563e474386ac80d2afc78e3690; no voters:
I20260619 21:14:56.701042 953 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:14:56.701546 953 raft_consensus.cc:686] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:56.702334 953 consensus_queue.cc:237] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.703720 530 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 5.
I20260619 21:14:56.708811 428 catalog_manager.cc:5495] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 4 to 5, leader changed from 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 5 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:56.793493 966 raft_consensus.cc:979] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.794075 935 raft_consensus.cc:1067] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.795715 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:56.796336 604 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:56.796716 604 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:56.804811 604 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.806901 604 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:56.807557 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:56.807879 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:56.808161 530 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:56.808375 677 raft_consensus.cc:3032] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:14:56.808666 677 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:56.809437 677 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.810462 677 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:14:56.814257 530 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:56.815557 571 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:56.816457 953 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:14:56.816622 677 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 4.
I20260619 21:14:56.817515 953 raft_consensus.cc:686] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:56.818364 953 consensus_queue.cc:237] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.825594 428 catalog_manager.cc:5495] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 3 to 4, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 4 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:56.924682 966 raft_consensus.cc:979] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: : Instructing follower 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.925181 935 raft_consensus.cc:1067] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Signalling peer 18e29b563e474386ac80d2afc78e3690 to start an election
I20260619 21:14:56.926697 604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
from {username='slave'} at 127.0.0.1:41966
I20260619 21:14:56.927312 604 raft_consensus.cc:482] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:56.927701 604 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:56.935926 604 raft_consensus.cc:504] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.938174 604 leader_election.cc:288] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:56.938676 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:56.939229 530 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:56.939356 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52855f5047c43ffacf998eec63c1b38" candidate_uuid: "18e29b563e474386ac80d2afc78e3690" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:56.939944 677 raft_consensus.cc:3032] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:56.940217 677 raft_consensus.cc:729] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:56.941035 677 consensus_queue.cc:260] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.942234 677 raft_consensus.cc:3037] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:56.946375 530 raft_consensus.cc:2454] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 3.
I20260619 21:14:56.947274 571 leader_election.cc:302] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:56.948046 953 raft_consensus.cc:2781] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:14:56.948513 677 raft_consensus.cc:2454] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e29b563e474386ac80d2afc78e3690 in term 3.
I20260619 21:14:56.948592 953 raft_consensus.cc:686] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 3 LEADER]: Becoming Leader. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:56.949792 953 consensus_queue.cc:237] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:56.957635 428 catalog_manager.cc:5495] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 reported cstate change: term changed from 2 to 3, leader changed from 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3) to 18e29b563e474386ac80d2afc78e3690 (127.0.99.2). New cstate: current_term: 3 leader_uuid: "18e29b563e474386ac80d2afc78e3690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:57.014037 530 raft_consensus.cc:1261] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:57.015206 953 consensus_queue.cc:1035] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:14:57.029275 677 raft_consensus.cc:1261] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:14:57.034550 958 consensus_queue.cc:1035] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:14:57.047739 675 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260619 21:14:57.049844 953 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260619 21:14:57.067312 530 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260619 21:14:57.069952 953 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260619 21:14:57.161418 675 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260619 21:14:57.163398 972 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:14:57.177266 530 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260619 21:14:57.178706 953 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:14:57.211025 675 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:57.212435 953 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:14:57.215927 530 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:14:57.217254 953 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:14:57.491125 675 raft_consensus.cc:1261] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:57.492364 958 consensus_queue.cc:1035] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:57.501279 530 raft_consensus.cc:1261] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Refusing update from remote peer 18e29b563e474386ac80d2afc78e3690: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:14:57.503012 953 consensus_queue.cc:1035] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:14:59.425258 396 auto_leader_rebalancer-test.cc:303] The leader distribution is
I20260619 21:14:59.426059 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:14:59.431962 604 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:59.432593 604 raft_consensus.cc:595] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 LEADER]: Received request to transfer leadership to TS 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:59.434882 604 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:59.435277 604 raft_consensus.cc:595] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 LEADER]: Received request to transfer leadership to TS 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:59.437175 604 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:59.437565 604 raft_consensus.cc:595] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Received request to transfer leadership to TS 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:59.439373 604 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:59.439754 604 raft_consensus.cc:595] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Received request to transfer leadership to TS 000d8010fc7a4d3ca6fced1e06386c3f
I20260619 21:14:59.441624 604 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d82359fd768143449f4fe443d1879af6"
dest_uuid: "18e29b563e474386ac80d2afc78e3690"
mode: GRACEFUL
new_leader_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:41990
I20260619 21:14:59.442021 604 raft_consensus.cc:595] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Received request to transfer leadership to TS 86439c50df6044b7be9eccfcf6066c1a
I20260619 21:14:59.442801 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260619 21:14:59.443187 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:14:59.505462 953 raft_consensus.cc:979] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:59.505956 958 raft_consensus.cc:1067] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Signalling peer 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:59.506953 958 raft_consensus.cc:979] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:59.507376 958 raft_consensus.cc:1067] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 LEADER]: Signalling peer 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:59.507427 528 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82359fd768143449f4fe443d1879af6"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:36814
I20260619 21:14:59.507946 528 raft_consensus.cc:482] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:59.508217 528 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:59.508780 530 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:36814
I20260619 21:14:59.509253 530 raft_consensus.cc:482] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:59.509503 530 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:14:59.513110 528 raft_consensus.cc:504] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.514155 530 raft_consensus.cc:504] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.515810 530 leader_election.cc:288] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 7 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:59.516088 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:59.516624 677 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:59.516736 602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:59.517304 602 raft_consensus.cc:3032] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 LEADER]: Stepping down as leader of term 6
I20260619 21:14:59.517491 602 raft_consensus.cc:729] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:59.515235 528 leader_election.cc:288] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 5 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:59.517992 675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9deb38fccd1f42acaaec7a7b57b3fc5e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:59.518033 602 consensus_queue.cc:260] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.518615 675 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:14:59.519368 602 raft_consensus.cc:3037] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:14:59.519518 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82359fd768143449f4fe443d1879af6" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:59.520082 604 raft_consensus.cc:3032] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:14:59.520361 604 raft_consensus.cc:729] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:59.521102 604 consensus_queue.cc:260] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.522394 604 raft_consensus.cc:3037] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:59.522778 677 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 5.
I20260619 21:14:59.525161 675 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 7.
I20260619 21:14:59.525799 494 leader_election.cc:302] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:59.526335 602 raft_consensus.cc:2454] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 7.
I20260619 21:14:59.527550 495 leader_election.cc:302] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e29b563e474386ac80d2afc78e3690, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:59.527809 985 raft_consensus.cc:2781] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:14:59.530099 986 raft_consensus.cc:2781] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 7 FOLLOWER]: Leader election won for term 7
I20260619 21:14:59.533797 986 raft_consensus.cc:686] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 7 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:59.535295 604 raft_consensus.cc:2454] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 5.
I20260619 21:14:59.535374 986 consensus_queue.cc:237] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.537456 985 raft_consensus.cc:686] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:59.538218 985 consensus_queue.cc:237] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.546741 426 catalog_manager.cc:5495] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 4 to 5, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 5 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:59.547942 426 catalog_manager.cc:5495] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 6 to 7, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 7 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:59.553297 953 raft_consensus.cc:979] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:59.553772 958 raft_consensus.cc:1067] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Signalling peer 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:59.555629 677 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:57364
I20260619 21:14:59.556041 677 raft_consensus.cc:482] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:59.556305 677 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:59.563257 677 raft_consensus.cc:504] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.565989 677 leader_election.cc:288] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 5 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:59.567099 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:59.567242 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15841e7a6be4a1f84f7e64f03f6b0e8" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:59.567713 530 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:59.567770 604 raft_consensus.cc:3032] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:14:59.568185 604 raft_consensus.cc:729] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:59.568840 604 consensus_queue.cc:260] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.569895 604 raft_consensus.cc:3037] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:14:59.573642 530 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 5.
I20260619 21:14:59.574254 604 raft_consensus.cc:2454] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 5.
I20260619 21:14:59.575065 645 leader_election.cc:302] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:59.576064 935 raft_consensus.cc:2781] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:14:59.577647 935 raft_consensus.cc:686] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:59.578665 935 consensus_queue.cc:237] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.586725 426 catalog_manager.cc:5495] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 4 to 5, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 5 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:59.620414 953 raft_consensus.cc:979] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:59.620851 958 raft_consensus.cc:1067] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Signalling peer 000d8010fc7a4d3ca6fced1e06386c3f to start an election
I20260619 21:14:59.622201 677 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41"
dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
from {username='slave'} at 127.0.0.1:57364
I20260619 21:14:59.622628 677 raft_consensus.cc:482] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:59.622884 677 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:59.628417 677 raft_consensus.cc:504] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.629678 677 leader_election.cc:288] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Requested vote from peers 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099), 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541)
I20260619 21:14:59.630785 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
I20260619 21:14:59.630873 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7a148bc32a405ba2dbf5c5e78eec41" candidate_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:59.631256 530 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:59.631358 604 raft_consensus.cc:3032] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:14:59.631716 604 raft_consensus.cc:729] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:59.632296 604 consensus_queue.cc:260] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.633170 604 raft_consensus.cc:3037] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:14:59.635661 530 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:59.636571 645 leader_election.cc:302] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:59.637583 604 raft_consensus.cc:2454] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 000d8010fc7a4d3ca6fced1e06386c3f in term 3.
I20260619 21:14:59.637595 935 raft_consensus.cc:2781] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:14:59.638206 935 raft_consensus.cc:686] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Becoming Leader. State: Replica: 000d8010fc7a4d3ca6fced1e06386c3f, State: Running, Role: LEADER
I20260619 21:14:59.639222 935 consensus_queue.cc:237] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.647405 426 catalog_manager.cc:5495] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f reported cstate change: term changed from 2 to 3, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3). New cstate: current_term: 3 leader_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260619 21:14:59.717267 953 raft_consensus.cc:979] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: : Instructing follower 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:59.717694 958 raft_consensus.cc:1067] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 LEADER]: Signalling peer 86439c50df6044b7be9eccfcf6066c1a to start an election
I20260619 21:14:59.719033 530 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e"
dest_uuid: "86439c50df6044b7be9eccfcf6066c1a"
from {username='slave'} at 127.0.0.1:36814
I20260619 21:14:59.719436 530 raft_consensus.cc:482] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:14:59.719664 530 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:14:59.724068 530 raft_consensus.cc:504] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.725389 530 leader_election.cc:288] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 6 election: Requested vote from peers 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541), 000d8010fc7a4d3ca6fced1e06386c3f (127.0.99.3:37523)
I20260619 21:14:59.726297 604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "18e29b563e474386ac80d2afc78e3690"
I20260619 21:14:59.726325 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cd5249877784e9596d4250a1d88d49e" candidate_uuid: "86439c50df6044b7be9eccfcf6066c1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "000d8010fc7a4d3ca6fced1e06386c3f"
I20260619 21:14:59.726833 604 raft_consensus.cc:3032] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:14:59.726913 677 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:14:59.727181 604 raft_consensus.cc:729] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 18e29b563e474386ac80d2afc78e3690, State: Running, Role: LEADER
I20260619 21:14:59.727837 604 consensus_queue.cc:260] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.728919 604 raft_consensus.cc:3037] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:14:59.732960 677 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 6.
I20260619 21:14:59.733860 494 leader_election.cc:302] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000d8010fc7a4d3ca6fced1e06386c3f, 86439c50df6044b7be9eccfcf6066c1a; no voters:
I20260619 21:14:59.734460 985 raft_consensus.cc:2781] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Leader election won for term 6
I20260619 21:14:59.735049 604 raft_consensus.cc:2454] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86439c50df6044b7be9eccfcf6066c1a in term 6.
I20260619 21:14:59.735105 985 raft_consensus.cc:686] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 6 LEADER]: Becoming Leader. State: Replica: 86439c50df6044b7be9eccfcf6066c1a, State: Running, Role: LEADER
I20260619 21:14:59.736174 985 consensus_queue.cc:237] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } }
I20260619 21:14:59.744592 426 catalog_manager.cc:5495] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a reported cstate change: term changed from 5 to 6, leader changed from 18e29b563e474386ac80d2afc78e3690 (127.0.99.2) to 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1). New cstate: current_term: 6 leader_uuid: "86439c50df6044b7be9eccfcf6066c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260619 21:14:59.935099 604 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 7 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260619 21:14:59.936224 985 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 21:14:59.948269 677 raft_consensus.cc:1261] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 7 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260619 21:14:59.949666 985 consensus_queue.cc:1035] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260619 21:15:00.019086 677 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260619 21:15:00.020203 985 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260619 21:15:00.029420 604 raft_consensus.cc:1261] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260619 21:15:00.031653 987 consensus_queue.cc:1035] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260619 21:15:00.046416 530 raft_consensus.cc:1261] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:00.047683 935 consensus_queue.cc:1035] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:00.058825 604 raft_consensus.cc:1261] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:00.061461 530 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260619 21:15:00.061439 995 consensus_queue.cc:1035] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:15:00.063906 1007 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86439c50df6044b7be9eccfcf6066c1a" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260619 21:15:00.080708 604 raft_consensus.cc:1261] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Refusing update from remote peer 000d8010fc7a4d3ca6fced1e06386c3f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260619 21:15:00.082283 995 consensus_queue.cc:1035] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260619 21:15:00.119510 677 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260619 21:15:00.120739 987 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "000d8010fc7a4d3ca6fced1e06386c3f" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260619 21:15:00.134501 604 raft_consensus.cc:1261] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Refusing update from remote peer 86439c50df6044b7be9eccfcf6066c1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260619 21:15:00.136009 987 consensus_queue.cc:1035] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e29b563e474386ac80d2afc78e3690" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260619 21:15:00.443969 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:00.448858 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:00.449363 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:01.450234 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:01.455890 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:01.456297 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:02.457222 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:02.462060 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:02.462532 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:03.463335 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:03.467953 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:03.468302 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:04.469233 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:04.474002 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:04.474365 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260619 21:15:05.053582 495 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:33541, user_credentials={real_user=slave}} blocked reactor thread for 202029us
I20260619 21:15:05.525178 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:05.530103 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:05.530514 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260619 21:15:05.880551 459 debug-util.cc:398] Leaking SignalData structure 0x7b0800060060 after lost signal to thread 399
W20260619 21:15:05.881727 459 debug-util.cc:398] Leaking SignalData structure 0x7b0800060740 after lost signal to thread 462
W20260619 21:15:05.882752 459 debug-util.cc:398] Leaking SignalData structure 0x7b08001c60e0 after lost signal to thread 555
W20260619 21:15:05.883562 459 debug-util.cc:398] Leaking SignalData structure 0x7b0800115100 after lost signal to thread 630
W20260619 21:15:05.884356 459 debug-util.cc:398] Leaking SignalData structure 0x7b0800131180 after lost signal to thread 704
I20260619 21:15:06.531237 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:06.536104 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:06.536486 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:07.537369 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:07.543712 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:07.544214 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:08.545007 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:08.549839 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:08.550276 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:09.580621 396 auto_leader_rebalancer-test.cc:314] The leader distribution is
I20260619 21:15:09.581384 396 tablet_server.cc:178] TabletServer@127.0.99.1:0 shutting down...
I20260619 21:15:09.609530 396 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260619 21:15:09.610252 396 tablet_replica.cc:330] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.611095 396 raft_consensus.cc:2229] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.611743 396 raft_consensus.cc:2258] T aeb7af6b7887456382ec990dec4c06d6 P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.614189 396 tablet_replica.cc:330] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.614691 396 raft_consensus.cc:2229] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.615175 396 raft_consensus.cc:2258] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 86439c50df6044b7be9eccfcf6066c1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.616901 396 tablet_replica.cc:330] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.617453 396 raft_consensus.cc:2229] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.617841 396 raft_consensus.cc:2258] T e48c29301ab04ff5bf0c2e96e3196ac1 P 86439c50df6044b7be9eccfcf6066c1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.619360 396 tablet_replica.cc:330] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.620124 396 raft_consensus.cc:2229] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.620699 396 raft_consensus.cc:2258] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Raft consensus is shut down!
W20260619 21:15:09.621830 571 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260619 21:15:09.622939 396 tablet_replica.cc:330] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.623521 396 raft_consensus.cc:2229] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.624027 396 raft_consensus.cc:2258] T d52855f5047c43ffacf998eec63c1b38 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260619 21:15:09.625115 571 consensus_peers.cc:487] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 -> Peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Couldn't send request to peer 86439c50df6044b7be9eccfcf6066c1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:09.625887 396 tablet_replica.cc:330] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.626389 396 raft_consensus.cc:2229] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 7 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.627209 396 raft_consensus.cc:2258] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 86439c50df6044b7be9eccfcf6066c1a [term 7 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.629066 396 tablet_replica.cc:330] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.629519 396 raft_consensus.cc:2229] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 6 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.630347 396 raft_consensus.cc:2258] T 1cd5249877784e9596d4250a1d88d49e P 86439c50df6044b7be9eccfcf6066c1a [term 6 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.631920 396 tablet_replica.cc:330] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.632335 396 raft_consensus.cc:2229] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 5 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.633193 396 raft_consensus.cc:2258] T d82359fd768143449f4fe443d1879af6 P 86439c50df6044b7be9eccfcf6066c1a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.634737 396 tablet_replica.cc:330] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a: stopping tablet replica
I20260619 21:15:09.635166 396 raft_consensus.cc:2229] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.635691 396 raft_consensus.cc:2258] T cd7a148bc32a405ba2dbf5c5e78eec41 P 86439c50df6044b7be9eccfcf6066c1a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.722622 396 tablet_server.cc:194] TabletServer@127.0.99.1:0 shutdown complete.
I20260619 21:15:09.743170 396 tablet_server.cc:178] TabletServer@127.0.99.2:0 shutting down...
I20260619 21:15:09.772500 396 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260619 21:15:09.773216 396 tablet_replica.cc:330] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.773892 396 raft_consensus.cc:2229] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.774833 396 raft_consensus.cc:2258] T aeb7af6b7887456382ec990dec4c06d6 P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.776623 396 tablet_replica.cc:330] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.777251 396 raft_consensus.cc:2229] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.777731 396 raft_consensus.cc:2258] T d82359fd768143449f4fe443d1879af6 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.779575 396 tablet_replica.cc:330] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.780105 396 raft_consensus.cc:2229] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.780540 396 raft_consensus.cc:2258] T 1cd5249877784e9596d4250a1d88d49e P 18e29b563e474386ac80d2afc78e3690 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.782181 396 tablet_replica.cc:330] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.782655 396 raft_consensus.cc:2229] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.783070 396 raft_consensus.cc:2258] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 18e29b563e474386ac80d2afc78e3690 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.784711 396 tablet_replica.cc:330] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.785228 396 raft_consensus.cc:2229] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.785624 396 raft_consensus.cc:2258] T e48c29301ab04ff5bf0c2e96e3196ac1 P 18e29b563e474386ac80d2afc78e3690 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.787187 396 tablet_replica.cc:330] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.787626 396 raft_consensus.cc:2229] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.788365 396 raft_consensus.cc:2258] T d52855f5047c43ffacf998eec63c1b38 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.789847 396 tablet_replica.cc:330] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.790274 396 raft_consensus.cc:2229] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.790693 396 raft_consensus.cc:2258] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 18e29b563e474386ac80d2afc78e3690 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.792207 396 tablet_replica.cc:330] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.792636 396 raft_consensus.cc:2229] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.793097 396 raft_consensus.cc:2258] T cd7a148bc32a405ba2dbf5c5e78eec41 P 18e29b563e474386ac80d2afc78e3690 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.794675 396 tablet_replica.cc:330] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690: stopping tablet replica
I20260619 21:15:09.795079 396 raft_consensus.cc:2229] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 4 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.795840 396 raft_consensus.cc:2258] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 18e29b563e474386ac80d2afc78e3690 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.849007 396 tablet_server.cc:194] TabletServer@127.0.99.2:0 shutdown complete.
W20260619 21:15:09.866096 644 consensus_peers.cc:487] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f -> Peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Couldn't send request to peer 18e29b563e474386ac80d2afc78e3690. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.2:33541: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:09.869168 396 tablet_server.cc:178] TabletServer@127.0.99.3:0 shutting down...
W20260619 21:15:09.877420 644 consensus_peers.cc:487] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f -> Peer 18e29b563e474386ac80d2afc78e3690 (127.0.99.2:33541): Couldn't send request to peer 18e29b563e474386ac80d2afc78e3690. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.2:33541: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260619 21:15:09.894026 645 consensus_peers.cc:487] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f -> Peer 86439c50df6044b7be9eccfcf6066c1a (127.0.99.1:40099): Couldn't send request to peer 86439c50df6044b7be9eccfcf6066c1a. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.1:40099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:09.901238 396 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260619 21:15:09.901871 396 tablet_replica.cc:330] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.902472 396 raft_consensus.cc:2229] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.902911 396 raft_consensus.cc:2258] T 8e14f261c84d4d6a8c9f42f4097f8ecb P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.904628 396 tablet_replica.cc:330] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.905186 396 raft_consensus.cc:2229] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.905679 396 raft_consensus.cc:2258] T 1cd5249877784e9596d4250a1d88d49e P 000d8010fc7a4d3ca6fced1e06386c3f [term 6 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.907171 396 tablet_replica.cc:330] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.907626 396 raft_consensus.cc:2229] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 7 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.908041 396 raft_consensus.cc:2258] T 9deb38fccd1f42acaaec7a7b57b3fc5e P 000d8010fc7a4d3ca6fced1e06386c3f [term 7 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.909659 396 tablet_replica.cc:330] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.910128 396 raft_consensus.cc:2229] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.910562 396 raft_consensus.cc:2258] T aeb7af6b7887456382ec990dec4c06d6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.912026 396 tablet_replica.cc:330] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.912511 396 raft_consensus.cc:2229] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.913324 396 raft_consensus.cc:2258] T cd7a148bc32a405ba2dbf5c5e78eec41 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.915405 396 tablet_replica.cc:330] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.916136 396 raft_consensus.cc:2229] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.916625 396 raft_consensus.cc:2258] T d82359fd768143449f4fe443d1879af6 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.918386 396 tablet_replica.cc:330] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.918756 396 raft_consensus.cc:2229] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.919364 396 raft_consensus.cc:2258] T e48c29301ab04ff5bf0c2e96e3196ac1 P 000d8010fc7a4d3ca6fced1e06386c3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.920771 396 tablet_replica.cc:330] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.921196 396 raft_consensus.cc:2229] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 LEADER]: Raft consensus shutting down.
I20260619 21:15:09.921913 396 raft_consensus.cc:2258] T b15841e7a6be4a1f84f7e64f03f6b0e8 P 000d8010fc7a4d3ca6fced1e06386c3f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.923288 396 tablet_replica.cc:330] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f: stopping tablet replica
I20260619 21:15:09.923700 396 raft_consensus.cc:2229] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:09.924100 396 raft_consensus.cc:2258] T d52855f5047c43ffacf998eec63c1b38 P 000d8010fc7a4d3ca6fced1e06386c3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:09.968423 396 tablet_server.cc:194] TabletServer@127.0.99.3:0 shutdown complete.
I20260619 21:15:09.987035 396 master.cc:537] Master@127.0.99.62:32921 shutting down...
I20260619 21:15:10.050076 483 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260619 21:15:10.105381 396 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 1 LEADER]: Raft consensus shutting down.
I20260619 21:15:10.106139 396 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:10.106524 396 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5d773a50e1984526a6d2a8a5df258f5f: stopping tablet replica
I20260619 21:15:10.142673 396 master.cc:552] Master@127.0.99.62:32921 shutdown complete.
000d8010fc7a4d3ca6fced1e06386c3f 1
18e29b563e474386ac80d2afc78e3690 4
86439c50df6044b7be9eccfcf6066c1a 4
000d8010fc7a4d3ca6fced1e06386c3f 3
18e29b563e474386ac80d2afc78e3690 3
86439c50df6044b7be9eccfcf6066c1a 3
000d8010fc7a4d3ca6fced1e06386c3f 1
18e29b563e474386ac80d2afc78e3690 8
86439c50df6044b7be9eccfcf6066c1a 0
000d8010fc7a4d3ca6fced1e06386c3f 3
18e29b563e474386ac80d2afc78e3690 3
86439c50df6044b7be9eccfcf6066c1a 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (35327 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260619 21:15:10.186352 396 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.62:45773
I20260619 21:15:10.187413 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:10.193572 1027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:10.193696 1028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:10.194475 1030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:10.194896 396 server_base.cc:1034] running on GCE node
I20260619 21:15:10.196041 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:10.196218 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:10.196333 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903710196325 us; error 0 us; skew 500 ppm
I20260619 21:15:10.196784 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:10.203061 396 webserver.cc:460] Webserver started at http://127.0.99.62:35687/ using document root <none> and password file <none>
I20260619 21:15:10.203575 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:10.203778 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:10.204073 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:10.205380 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/master-0-root/instance:
uuid: "59415aedeaeb4d75ba16aa3d1bc61903"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.209802 396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 21:15:10.213176 1035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.214053 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260619 21:15:10.214474 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/master-0-root
uuid: "59415aedeaeb4d75ba16aa3d1bc61903"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.214749 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:10.235546 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:10.236835 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:10.276907 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.62:45773
I20260619 21:15:10.277060 1086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.62:45773 every 8 connection(s)
I20260619 21:15:10.280745 1087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.290351 1087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903: Bootstrap starting.
I20260619 21:15:10.294927 1087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:10.299031 1087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903: No bootstrap required, opened a new log
I20260619 21:15:10.301137 1087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" member_type: VOTER }
I20260619 21:15:10.301642 1087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:10.301916 1087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59415aedeaeb4d75ba16aa3d1bc61903, State: Initialized, Role: FOLLOWER
I20260619 21:15:10.302531 1087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" member_type: VOTER }
I20260619 21:15:10.302987 1087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 21:15:10.303226 1087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 21:15:10.303521 1087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:10.307420 1087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" member_type: VOTER }
I20260619 21:15:10.307878 1087 leader_election.cc:302] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59415aedeaeb4d75ba16aa3d1bc61903; no voters:
I20260619 21:15:10.309018 1087 leader_election.cc:288] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 21:15:10.309420 1090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:10.310878 1090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 1 LEADER]: Becoming Leader. State: Replica: 59415aedeaeb4d75ba16aa3d1bc61903, State: Running, Role: LEADER
I20260619 21:15:10.311470 1090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" member_type: VOTER }
I20260619 21:15:10.312027 1087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 21:15:10.316229 1092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59415aedeaeb4d75ba16aa3d1bc61903. Latest consensus state: current_term: 1 leader_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" member_type: VOTER } }
I20260619 21:15:10.316809 1092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [sys.catalog]: This master's current role is: LEADER
I20260619 21:15:10.317206 1091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59415aedeaeb4d75ba16aa3d1bc61903" member_type: VOTER } }
I20260619 21:15:10.317873 1091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [sys.catalog]: This master's current role is: LEADER
I20260619 21:15:10.324163 1097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 21:15:10.331350 1097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 21:15:10.332396 396 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260619 21:15:10.339846 1097 catalog_manager.cc:1347] Generated new cluster ID: cb77c80026454953bd268308aac5926d
I20260619 21:15:10.340096 1097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 21:15:10.369050 1097 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 21:15:10.370630 1097 catalog_manager.cc:1504] Loading token signing keys...
I20260619 21:15:10.384791 1097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903: Generated new TSK 0
I20260619 21:15:10.385450 1097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 21:15:10.398964 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:10.405427 1108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:10.405936 1109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:10.408506 396 server_base.cc:1034] running on GCE node
W20260619 21:15:10.408588 1111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:10.409696 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:10.409906 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:10.410074 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903710410044 us; error 0 us; skew 500 ppm
I20260619 21:15:10.410583 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:10.412875 396 webserver.cc:460] Webserver started at http://127.0.99.1:46495/ using document root <none> and password file <none>
I20260619 21:15:10.413534 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:10.413724 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:10.414083 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:10.415189 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-0-root/instance:
uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.419709 396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 21:15:10.422770 1116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.423527 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 21:15:10.423808 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-0-root
uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.424062 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:10.437705 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:10.438874 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:10.440184 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:15:10.442425 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:15:10.442579 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.442849 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:15:10.443022 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.484848 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.1:36165
I20260619 21:15:10.485013 1178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.1:36165 every 8 connection(s)
I20260619 21:15:10.489588 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:10.497942 1183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:10.499946 1184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:10.501363 1186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:10.501736 396 server_base.cc:1034] running on GCE node
I20260619 21:15:10.503564 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:10.503743 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:10.503947 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903710503916 us; error 0 us; skew 500 ppm
I20260619 21:15:10.504474 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:10.506173 1179 heartbeater.cc:347] Connected to a master server at 127.0.99.62:45773
I20260619 21:15:10.506547 1179 heartbeater.cc:464] Registering TS with master...
I20260619 21:15:10.507412 1179 heartbeater.cc:511] Master 127.0.99.62:45773 requested a full tablet report, sending...
I20260619 21:15:10.507465 396 webserver.cc:460] Webserver started at http://127.0.99.2:41159/ using document root <none> and password file <none>
I20260619 21:15:10.508167 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:10.508430 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:10.508706 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:10.509729 1052 ts_manager.cc:194] Registered new tserver with Master: e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165)
I20260619 21:15:10.509899 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-1-root/instance:
uuid: "0430d901e2f04acf93e0dfc23551ac1c"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.511720 1052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40834
I20260619 21:15:10.515291 396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 21:15:10.518829 1191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.519587 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 21:15:10.519868 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-1-root
uuid: "0430d901e2f04acf93e0dfc23551ac1c"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.520153 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:10.546892 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:10.548185 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:10.549734 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:15:10.551975 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:15:10.552192 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.552426 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:15:10.552590 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.595242 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.2:35879
I20260619 21:15:10.595327 1253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.2:35879 every 8 connection(s)
I20260619 21:15:10.600200 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:10.608824 1257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:10.613699 1258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:10.618678 1254 heartbeater.cc:347] Connected to a master server at 127.0.99.62:45773
W20260619 21:15:10.618906 1260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:10.619236 1254 heartbeater.cc:464] Registering TS with master...
I20260619 21:15:10.619630 396 server_base.cc:1034] running on GCE node
I20260619 21:15:10.620229 1254 heartbeater.cc:511] Master 127.0.99.62:45773 requested a full tablet report, sending...
I20260619 21:15:10.620641 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:10.620980 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:10.621201 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903710621182 us; error 0 us; skew 500 ppm
I20260619 21:15:10.621951 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:10.622558 1052 ts_manager.cc:194] Registered new tserver with Master: 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:10.624603 1052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40838
I20260619 21:15:10.625165 396 webserver.cc:460] Webserver started at http://127.0.99.3:46619/ using document root <none> and password file <none>
I20260619 21:15:10.625841 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:10.626155 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:10.626466 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:10.627784 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-2-root/instance:
uuid: "185ab65a08dd4d908675e66d19950d21"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.632647 396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260619 21:15:10.635999 1265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.636732 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 21:15:10.637068 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-2-root
uuid: "185ab65a08dd4d908675e66d19950d21"
format_stamp: "Formatted at 2026-06-19 21:15:10 on dist-test-slave-nlqf"
I20260619 21:15:10.637377 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781903674708472-396-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:10.662451 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:10.663724 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:10.665207 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:15:10.667583 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:15:10.667809 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.668059 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:15:10.668244 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:10.712102 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.3:44547
I20260619 21:15:10.712208 1327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.3:44547 every 8 connection(s)
I20260619 21:15:10.729895 1328 heartbeater.cc:347] Connected to a master server at 127.0.99.62:45773
I20260619 21:15:10.730475 1328 heartbeater.cc:464] Registering TS with master...
I20260619 21:15:10.731190 1328 heartbeater.cc:511] Master 127.0.99.62:45773 requested a full tablet report, sending...
I20260619 21:15:10.733144 1052 ts_manager.cc:194] Registered new tserver with Master: 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:10.733438 396 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016894942s
I20260619 21:15:10.735306 1052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40840
I20260619 21:15:10.736268 396 test_util.cc:268] Using random seed: -910962828
I20260619 21:15:10.760522 1052 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40842:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 21:15:10.763545 1052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 21:15:10.837435 1141 tablet_service.cc:1467] Processing CreateTablet for tablet b5976ef9d5574abcb8ad2f3eb80e2567 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260619 21:15:10.838495 1143 tablet_service.cc:1467] Processing CreateTablet for tablet 7905cb353dda4e818a29a34f9f6bac0c (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260619 21:15:10.837427 1144 tablet_service.cc:1467] Processing CreateTablet for tablet 0b134eb5e43f4a43ab5bad911099ec60 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260619 21:15:10.838946 1141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5976ef9d5574abcb8ad2f3eb80e2567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.840839 1142 tablet_service.cc:1467] Processing CreateTablet for tablet 370269df5d5d434b8e3ff6a330cafe3b (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260619 21:15:10.841465 1144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b134eb5e43f4a43ab5bad911099ec60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.842235 1142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370269df5d5d434b8e3ff6a330cafe3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.843123 1143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7905cb353dda4e818a29a34f9f6bac0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.844658 1291 tablet_service.cc:1467] Processing CreateTablet for tablet 370269df5d5d434b8e3ff6a330cafe3b (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260619 21:15:10.845057 1219 tablet_service.cc:1467] Processing CreateTablet for tablet 0b134eb5e43f4a43ab5bad911099ec60 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260619 21:15:10.846060 1291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370269df5d5d434b8e3ff6a330cafe3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.845925 1292 tablet_service.cc:1467] Processing CreateTablet for tablet 7905cb353dda4e818a29a34f9f6bac0c (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260619 21:15:10.846454 1219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b134eb5e43f4a43ab5bad911099ec60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.844672 1293 tablet_service.cc:1467] Processing CreateTablet for tablet 0b134eb5e43f4a43ab5bad911099ec60 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260619 21:15:10.847239 1216 tablet_service.cc:1467] Processing CreateTablet for tablet b5976ef9d5574abcb8ad2f3eb80e2567 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260619 21:15:10.848186 1293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b134eb5e43f4a43ab5bad911099ec60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.848507 1216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5976ef9d5574abcb8ad2f3eb80e2567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.849305 1292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7905cb353dda4e818a29a34f9f6bac0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.850231 1218 tablet_service.cc:1467] Processing CreateTablet for tablet 7905cb353dda4e818a29a34f9f6bac0c (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260619 21:15:10.851516 1218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7905cb353dda4e818a29a34f9f6bac0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.854125 1217 tablet_service.cc:1467] Processing CreateTablet for tablet 370269df5d5d434b8e3ff6a330cafe3b (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260619 21:15:10.855387 1217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370269df5d5d434b8e3ff6a330cafe3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.863952 1290 tablet_service.cc:1467] Processing CreateTablet for tablet b5976ef9d5574abcb8ad2f3eb80e2567 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260619 21:15:10.865448 1290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5976ef9d5574abcb8ad2f3eb80e2567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.865988 1140 tablet_service.cc:1467] Processing CreateTablet for tablet e625e8cd81b24cdd8f9ee28de8e0ee9c (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260619 21:15:10.867661 1140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e625e8cd81b24cdd8f9ee28de8e0ee9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.880757 1215 tablet_service.cc:1467] Processing CreateTablet for tablet e625e8cd81b24cdd8f9ee28de8e0ee9c (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260619 21:15:10.882547 1215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e625e8cd81b24cdd8f9ee28de8e0ee9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.901041 1289 tablet_service.cc:1467] Processing CreateTablet for tablet e625e8cd81b24cdd8f9ee28de8e0ee9c (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260619 21:15:10.902876 1289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e625e8cd81b24cdd8f9ee28de8e0ee9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.907682 1139 tablet_service.cc:1467] Processing CreateTablet for tablet f86dbcdc0853415d8bd72bf06f596175 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260619 21:15:10.909503 1139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86dbcdc0853415d8bd72bf06f596175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.914736 1214 tablet_service.cc:1467] Processing CreateTablet for tablet f86dbcdc0853415d8bd72bf06f596175 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260619 21:15:10.916522 1214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86dbcdc0853415d8bd72bf06f596175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.922518 1138 tablet_service.cc:1467] Processing CreateTablet for tablet 367501a407f64ec89b6080d0c2277fea (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260619 21:15:10.923435 1288 tablet_service.cc:1467] Processing CreateTablet for tablet f86dbcdc0853415d8bd72bf06f596175 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260619 21:15:10.923950 1138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 367501a407f64ec89b6080d0c2277fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.929672 1287 tablet_service.cc:1467] Processing CreateTablet for tablet 367501a407f64ec89b6080d0c2277fea (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260619 21:15:10.933997 1216 tablet_service.cc:1467] Processing CreateTablet for tablet 367501a407f64ec89b6080d0c2277fea (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260619 21:15:10.951221 1216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 367501a407f64ec89b6080d0c2277fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.955876 1343 tablet_bootstrap.cc:492] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:10.949215 1288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86dbcdc0853415d8bd72bf06f596175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.960489 1344 tablet_bootstrap.cc:492] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:10.961220 1287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 367501a407f64ec89b6080d0c2277fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.965062 1345 tablet_bootstrap.cc:492] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:10.974876 1345 tablet_bootstrap.cc:654] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:10.981967 1140 tablet_service.cc:1467] Processing CreateTablet for tablet b5e0108caeb6467698431893252bb550 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260619 21:15:10.983245 1140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5e0108caeb6467698431893252bb550. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:10.990074 1219 tablet_service.cc:1467] Processing CreateTablet for tablet b5e0108caeb6467698431893252bb550 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260619 21:15:10.991734 1219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5e0108caeb6467698431893252bb550. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.002081 1289 tablet_service.cc:1467] Processing CreateTablet for tablet b5e0108caeb6467698431893252bb550 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260619 21:15:11.002805 1143 tablet_service.cc:1467] Processing CreateTablet for tablet fbda230b058e4a9b8aa6d0dbc5ace417 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260619 21:15:11.003379 1289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5e0108caeb6467698431893252bb550. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.003512 1343 tablet_bootstrap.cc:654] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.004088 1143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbda230b058e4a9b8aa6d0dbc5ace417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.004297 1215 tablet_service.cc:1467] Processing CreateTablet for tablet fbda230b058e4a9b8aa6d0dbc5ace417 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260619 21:15:11.008581 1215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbda230b058e4a9b8aa6d0dbc5ace417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.015080 1144 tablet_service.cc:1467] Processing CreateTablet for tablet c498a73e8bdb476b975e8119817b81ff (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260619 21:15:11.016316 1144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c498a73e8bdb476b975e8119817b81ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.018283 1217 tablet_service.cc:1467] Processing CreateTablet for tablet c498a73e8bdb476b975e8119817b81ff (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260619 21:15:11.019407 1217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c498a73e8bdb476b975e8119817b81ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.022006 1292 tablet_service.cc:1467] Processing CreateTablet for tablet c498a73e8bdb476b975e8119817b81ff (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260619 21:15:11.027328 1345 tablet_bootstrap.cc:492] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.027812 1345 ts_tablet_manager.cc:1397] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.063s user 0.016s sys 0.007s
I20260619 21:15:11.030656 1345 raft_consensus.cc:348] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.031440 1345 raft_consensus.cc:374] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.031745 1345 raft_consensus.cc:729] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.024708 1292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c498a73e8bdb476b975e8119817b81ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.043085 1345 consensus_queue.cc:260] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.022964 1290 tablet_service.cc:1467] Processing CreateTablet for tablet fbda230b058e4a9b8aa6d0dbc5ace417 (DEFAULT_TABLE table=test-workload [id=34cdb8b1ac4b4ac48323520e8c7bd51f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260619 21:15:11.048187 1290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbda230b058e4a9b8aa6d0dbc5ace417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:11.023852 1344 tablet_bootstrap.cc:654] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.059111 1345 ts_tablet_manager.cc:1428] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.031s user 0.006s sys 0.005s
I20260619 21:15:11.059947 1179 heartbeater.cc:503] Master 127.0.99.62:45773 was elected leader, sending a full tablet report...
I20260619 21:15:11.062809 1343 tablet_bootstrap.cc:492] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.063301 1343 ts_tablet_manager.cc:1397] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.108s user 0.008s sys 0.006s
I20260619 21:15:11.064599 1345 tablet_bootstrap.cc:492] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.066751 1343 raft_consensus.cc:348] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.067504 1343 raft_consensus.cc:374] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.068524 1343 raft_consensus.cc:729] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.069367 1343 consensus_queue.cc:260] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.071137 1345 tablet_bootstrap.cc:654] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.075374 1343 ts_tablet_manager.cc:1428] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260619 21:15:11.076215 1328 heartbeater.cc:503] Master 127.0.99.62:45773 was elected leader, sending a full tablet report...
I20260619 21:15:11.082461 1343 tablet_bootstrap.cc:492] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.088804 1343 tablet_bootstrap.cc:654] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.095588 1344 tablet_bootstrap.cc:492] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.096171 1344 ts_tablet_manager.cc:1397] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.136s user 0.020s sys 0.013s
I20260619 21:15:11.099362 1344 raft_consensus.cc:348] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.100143 1344 raft_consensus.cc:374] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.100431 1344 raft_consensus.cc:729] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.101208 1345 tablet_bootstrap.cc:492] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.101136 1344 consensus_queue.cc:260] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.101719 1345 ts_tablet_manager.cc:1397] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.037s user 0.016s sys 0.008s
I20260619 21:15:11.105033 1345 raft_consensus.cc:348] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.105966 1345 raft_consensus.cc:374] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.106395 1343 tablet_bootstrap.cc:492] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.107017 1343 ts_tablet_manager.cc:1397] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.012s
I20260619 21:15:11.106336 1345 raft_consensus.cc:729] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.108197 1254 heartbeater.cc:503] Master 127.0.99.62:45773 was elected leader, sending a full tablet report...
I20260619 21:15:11.108449 1344 ts_tablet_manager.cc:1428] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260619 21:15:11.108237 1345 consensus_queue.cc:260] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.109478 1344 tablet_bootstrap.cc:492] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.110159 1343 raft_consensus.cc:348] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.110957 1343 raft_consensus.cc:374] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.111304 1343 raft_consensus.cc:729] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.112062 1343 consensus_queue.cc:260] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.112752 1345 ts_tablet_manager.cc:1428] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260619 21:15:11.113758 1345 tablet_bootstrap.cc:492] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.114514 1343 ts_tablet_manager.cc:1428] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 21:15:11.115496 1343 tablet_bootstrap.cc:492] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.117283 1344 tablet_bootstrap.cc:654] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.120275 1345 tablet_bootstrap.cc:654] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.122248 1343 tablet_bootstrap.cc:654] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.129402 1344 tablet_bootstrap.cc:492] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.129909 1344 ts_tablet_manager.cc:1397] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.008s
I20260619 21:15:11.131160 1345 tablet_bootstrap.cc:492] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.131623 1345 ts_tablet_manager.cc:1397] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.011s
I20260619 21:15:11.132568 1344 raft_consensus.cc:348] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.133387 1344 raft_consensus.cc:374] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.133666 1344 raft_consensus.cc:729] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.134178 1345 raft_consensus.cc:348] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.134902 1345 raft_consensus.cc:374] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.134464 1344 consensus_queue.cc:260] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.135206 1345 raft_consensus.cc:729] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.136737 1344 ts_tablet_manager.cc:1428] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260619 21:15:11.136484 1345 consensus_queue.cc:260] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.137661 1344 tablet_bootstrap.cc:492] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.141214 1345 ts_tablet_manager.cc:1428] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260619 21:15:11.141650 1343 tablet_bootstrap.cc:492] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.142168 1343 ts_tablet_manager.cc:1397] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.027s user 0.013s sys 0.011s
I20260619 21:15:11.142242 1345 tablet_bootstrap.cc:492] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.144994 1344 tablet_bootstrap.cc:654] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.145251 1343 raft_consensus.cc:348] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.146070 1343 raft_consensus.cc:374] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.146415 1343 raft_consensus.cc:729] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.147262 1343 consensus_queue.cc:260] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.149271 1345 tablet_bootstrap.cc:654] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.152227 1343 ts_tablet_manager.cc:1428] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260619 21:15:11.153183 1343 tablet_bootstrap.cc:492] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.157065 1344 tablet_bootstrap.cc:492] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.157618 1344 ts_tablet_manager.cc:1397] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.007s
I20260619 21:15:11.159487 1343 tablet_bootstrap.cc:654] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.160987 1344 raft_consensus.cc:348] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.161922 1344 raft_consensus.cc:374] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.162302 1344 raft_consensus.cc:729] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.163417 1344 consensus_queue.cc:260] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.167404 1344 ts_tablet_manager.cc:1428] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260619 21:15:11.167555 1345 tablet_bootstrap.cc:492] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.167992 1345 ts_tablet_manager.cc:1397] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.015s
I20260619 21:15:11.168322 1344 tablet_bootstrap.cc:492] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.170647 1345 raft_consensus.cc:348] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.171434 1345 raft_consensus.cc:374] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.171764 1345 raft_consensus.cc:729] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.172633 1345 consensus_queue.cc:260] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.174968 1344 tablet_bootstrap.cc:654] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.175038 1343 tablet_bootstrap.cc:492] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.175683 1343 ts_tablet_manager.cc:1397] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.001s
I20260619 21:15:11.177783 1345 ts_tablet_manager.cc:1428] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260619 21:15:11.179060 1345 tablet_bootstrap.cc:492] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.178637 1343 raft_consensus.cc:348] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.179550 1343 raft_consensus.cc:374] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.179901 1343 raft_consensus.cc:729] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.180791 1343 consensus_queue.cc:260] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.185979 1343 ts_tablet_manager.cc:1428] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260619 21:15:11.186700 1345 tablet_bootstrap.cc:654] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.186756 1344 tablet_bootstrap.cc:492] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.187017 1343 tablet_bootstrap.cc:492] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.187387 1344 ts_tablet_manager.cc:1397] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.005s
I20260619 21:15:11.190383 1344 raft_consensus.cc:348] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.191901 1344 raft_consensus.cc:374] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.192278 1344 raft_consensus.cc:729] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.196331 1343 tablet_bootstrap.cc:654] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.193259 1344 consensus_queue.cc:260] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.199407 1344 ts_tablet_manager.cc:1428] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260619 21:15:11.200681 1344 tablet_bootstrap.cc:492] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.209120 1343 tablet_bootstrap.cc:492] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.209781 1343 ts_tablet_manager.cc:1397] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.013s
I20260619 21:15:11.208865 1344 tablet_bootstrap.cc:654] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.212013 1345 tablet_bootstrap.cc:492] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.212522 1345 ts_tablet_manager.cc:1397] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.034s user 0.016s sys 0.015s
I20260619 21:15:11.212195 1343 raft_consensus.cc:348] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.213038 1343 raft_consensus.cc:374] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.213316 1343 raft_consensus.cc:729] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.214320 1343 consensus_queue.cc:260] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.215400 1350 raft_consensus.cc:482] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.215515 1345 raft_consensus.cc:348] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.216251 1344 tablet_bootstrap.cc:492] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.215900 1350 raft_consensus.cc:504] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.216439 1345 raft_consensus.cc:374] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.216712 1344 ts_tablet_manager.cc:1397] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260619 21:15:11.216790 1345 raft_consensus.cc:729] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.217787 1345 consensus_queue.cc:260] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.219581 1344 raft_consensus.cc:348] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.220340 1344 raft_consensus.cc:374] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.220633 1344 raft_consensus.cc:729] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.221627 1344 consensus_queue.cc:260] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.222587 1345 ts_tablet_manager.cc:1428] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260619 21:15:11.223511 1345 tablet_bootstrap.cc:492] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
W20260619 21:15:11.219928 1329 tablet.cc:2369] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 21:15:11.228219 1344 ts_tablet_manager.cc:1428] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.011s user 0.001s sys 0.005s
I20260619 21:15:11.229257 1344 tablet_bootstrap.cc:492] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.230103 1348 raft_consensus.cc:482] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.230574 1348 raft_consensus.cc:504] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.235602 1344 tablet_bootstrap.cc:654] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.241916 1348 leader_election.cc:288] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.264667 1345 tablet_bootstrap.cc:654] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.265509 1343 ts_tablet_manager.cc:1428] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.055s user 0.015s sys 0.036s
I20260619 21:15:11.266703 1343 tablet_bootstrap.cc:492] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.271888 1350 leader_election.cc:288] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:11.274065 1343 tablet_bootstrap.cc:654] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.283859 1345 tablet_bootstrap.cc:492] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.284536 1345 ts_tablet_manager.cc:1397] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.061s user 0.006s sys 0.004s
I20260619 21:15:11.288798 1353 raft_consensus.cc:482] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.289319 1353 raft_consensus.cc:504] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.291996 1353 leader_election.cc:288] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.306212 1345 raft_consensus.cc:348] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.307250 1345 raft_consensus.cc:374] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.307605 1345 raft_consensus.cc:729] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.310070 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.311008 1154 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 0.
I20260619 21:15:11.312858 1269 leader_election.cc:302] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.314208 1350 raft_consensus.cc:2781] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.314708 1350 raft_consensus.cc:482] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.315351 1350 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.318219 1344 tablet_bootstrap.cc:492] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.318673 1344 ts_tablet_manager.cc:1397] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.090s user 0.032s sys 0.034s
I20260619 21:15:11.322373 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.323361 1229 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 0.
I20260619 21:15:11.324851 1344 raft_consensus.cc:348] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
W20260619 21:15:11.325387 1100 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:11.325659 1344 raft_consensus.cc:374] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.325945 1344 raft_consensus.cc:729] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.326572 1344 consensus_queue.cc:260] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.327055 1118 leader_election.cc:302] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.327952 1348 raft_consensus.cc:2781] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.328419 1348 raft_consensus.cc:482] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.328819 1348 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.336294 1344 ts_tablet_manager.cc:1428] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260619 21:15:11.337246 1344 tablet_bootstrap.cc:492] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.339221 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.339941 1303 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 0.
I20260619 21:15:11.346192 1345 consensus_queue.cc:260] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.367170 1344 tablet_bootstrap.cc:654] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.369412 1350 raft_consensus.cc:504] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.373281 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.374738 1154 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.376144 1350 leader_election.cc:288] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:11.349496 1348 raft_consensus.cc:504] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.377314 1345 ts_tablet_manager.cc:1428] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.092s user 0.018s sys 0.017s
I20260619 21:15:11.379037 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:11.379598 1229 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.386606 1348 leader_election.cc:288] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.387254 1348 raft_consensus.cc:482] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.387701 1348 raft_consensus.cc:504] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.389768 1348 leader_election.cc:288] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.390766 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.391587 1228 raft_consensus.cc:2454] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 0.
I20260619 21:15:11.392959 1118 leader_election.cc:302] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.394743 1374 raft_consensus.cc:2781] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.395077 1229 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 1.
I20260619 21:15:11.396624 1154 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 1.
I20260619 21:15:11.395126 1343 tablet_bootstrap.cc:492] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.396816 1118 leader_election.cc:302] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.397722 1348 raft_consensus.cc:2781] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.397950 1343 ts_tablet_manager.cc:1397] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.131s user 0.028s sys 0.071s
I20260619 21:15:11.398182 1348 raft_consensus.cc:686] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:11.397946 1269 leader_election.cc:302] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.399083 1348 consensus_queue.cc:237] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.401918 1343 raft_consensus.cc:348] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.402781 1343 raft_consensus.cc:374] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.403029 1343 raft_consensus.cc:729] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.404901 1344 tablet_bootstrap.cc:492] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.405155 1350 raft_consensus.cc:2781] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.405943 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.406275 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:11.406761 1229 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 0.
I20260619 21:15:11.406839 1350 raft_consensus.cc:686] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Becoming Leader. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:11.405483 1344 ts_tablet_manager.cc:1397] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.068s user 0.010s sys 0.005s
I20260619 21:15:11.408108 1350 consensus_queue.cc:237] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.410895 1344 raft_consensus.cc:348] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.411744 1344 raft_consensus.cc:374] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.412122 1344 raft_consensus.cc:729] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.412484 1345 tablet_bootstrap.cc:492] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.413125 1344 consensus_queue.cc:260] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.395087 1374 raft_consensus.cc:482] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.415462 1374 raft_consensus.cc:3037] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.417572 1343 consensus_queue.cc:260] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.420157 1343 ts_tablet_manager.cc:1428] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.022s user 0.004s sys 0.000s
I20260619 21:15:11.420903 1344 ts_tablet_manager.cc:1428] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.013s user 0.002s sys 0.005s
I20260619 21:15:11.422751 1344 tablet_bootstrap.cc:492] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.422757 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.423766 1154 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.425624 1195 leader_election.cc:302] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.426724 1353 raft_consensus.cc:2781] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.427099 1353 raft_consensus.cc:482] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.427455 1353 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.428624 1343 tablet_bootstrap.cc:492] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.429286 1379 raft_consensus.cc:482] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.429734 1379 raft_consensus.cc:504] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.433012 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.433666 1229 raft_consensus.cc:2454] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 0.
I20260619 21:15:11.434855 1118 leader_election.cc:302] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.435825 1343 tablet_bootstrap.cc:654] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.439815 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.440531 1303 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.441314 1379 leader_election.cc:288] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.442600 1379 raft_consensus.cc:2781] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.442922 1379 raft_consensus.cc:482] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.443243 1379 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.455834 1301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.458168 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.458411 1379 raft_consensus.cc:504] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.459177 1302 raft_consensus.cc:2454] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 0.
I20260619 21:15:11.460481 1345 tablet_bootstrap.cc:654] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.461176 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:11.461750 1229 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.465607 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.466382 1303 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.466290 1374 raft_consensus.cc:504] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.469187 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:11.469743 1228 raft_consensus.cc:3037] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:11.471208 1117 leader_election.cc:341] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.473559 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.474520 1302 raft_consensus.cc:3037] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.478286 1229 raft_consensus.cc:2454] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 1.
I20260619 21:15:11.479658 1118 leader_election.cc:302] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.481109 1376 raft_consensus.cc:2781] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.463855 1050 catalog_manager.cc:5495] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 185ab65a08dd4d908675e66d19950d21 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "185ab65a08dd4d908675e66d19950d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:11.488685 1228 raft_consensus.cc:2454] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 1.
I20260619 21:15:11.497818 1118 leader_election.cc:302] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.497177 1051 catalog_manager.cc:5495] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 0 to 1, leader changed from <none> to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.498368 1350 raft_consensus.cc:482] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.498782 1348 raft_consensus.cc:2781] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.499308 1348 raft_consensus.cc:686] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:11.498910 1350 raft_consensus.cc:504] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.502164 1376 raft_consensus.cc:686] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:11.502673 1350 leader_election.cc:288] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:11.497686 1353 raft_consensus.cc:504] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.502918 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.503094 1376 consensus_queue.cc:237] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
W20260619 21:15:11.504323 1269 leader_election.cc:341] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.505473 1353 leader_election.cc:288] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.505754 1301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.506034 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.506534 1301 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.506707 1154 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.500250 1348 consensus_queue.cc:237] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.514832 1379 leader_election.cc:288] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.515365 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.515812 1154 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
W20260619 21:15:11.517732 1117 leader_election.cc:341] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.518249 1345 tablet_bootstrap.cc:492] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.518882 1345 ts_tablet_manager.cc:1397] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.107s user 0.018s sys 0.016s
I20260619 21:15:11.521592 1195 leader_election.cc:302] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.522637 1353 raft_consensus.cc:2781] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.523444 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.471424 1374 leader_election.cc:288] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 1 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
W20260619 21:15:11.525398 1267 leader_election.cc:341] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.525884 1267 leader_election.cc:302] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21; no voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:11.527341 1350 raft_consensus.cc:2726] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:11.533090 1303 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 1.
I20260619 21:15:11.536154 1353 raft_consensus.cc:686] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:11.535388 1344 tablet_bootstrap.cc:654] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.537348 1353 consensus_queue.cc:237] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.535985 1301 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:11.531083 1302 raft_consensus.cc:2454] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 1.
I20260619 21:15:11.548243 1387 raft_consensus.cc:482] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.549226 1387 raft_consensus.cc:504] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.550629 1345 raft_consensus.cc:348] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.551715 1345 raft_consensus.cc:374] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.552116 1345 raft_consensus.cc:729] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.552884 1387 leader_election.cc:288] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.554333 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.554972 1302 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.555774 1343 tablet_bootstrap.cc:492] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.556267 1343 ts_tablet_manager.cc:1397] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.128s user 0.061s sys 0.045s
I20260619 21:15:11.556232 1192 leader_election.cc:302] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:11.552760 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.557354 1154 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.553226 1345 consensus_queue.cc:260] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.558650 1344 tablet_bootstrap.cc:492] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.559171 1344 ts_tablet_manager.cc:1397] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.137s user 0.014s sys 0.002s
I20260619 21:15:11.559032 1343 raft_consensus.cc:348] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.559878 1343 raft_consensus.cc:374] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.559000 1387 raft_consensus.cc:2781] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.560163 1343 raft_consensus.cc:729] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.560597 1387 raft_consensus.cc:482] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.561020 1387 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.561275 1343 consensus_queue.cc:260] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.562832 1344 raft_consensus.cc:348] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.563701 1344 raft_consensus.cc:374] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.564011 1344 raft_consensus.cc:729] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.564962 1344 consensus_queue.cc:260] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.560153 1345 ts_tablet_manager.cc:1428] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.041s user 0.008s sys 0.001s
I20260619 21:15:11.564121 1343 ts_tablet_manager.cc:1428] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260619 21:15:11.568073 1345 tablet_bootstrap.cc:492] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.568491 1343 tablet_bootstrap.cc:492] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.569995 1387 raft_consensus.cc:504] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.571841 1344 ts_tablet_manager.cc:1428] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.012s user 0.005s sys 0.002s
I20260619 21:15:11.573172 1344 tablet_bootstrap.cc:492] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.575834 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.576607 1302 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.574954 1387 leader_election.cc:288] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.586176 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.586826 1154 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.588663 1344 tablet_bootstrap.cc:654] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.598301 1154 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:11.599664 1195 leader_election.cc:302] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.600868 1353 raft_consensus.cc:2781] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.600965 1388 raft_consensus.cc:482] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.601502 1353 raft_consensus.cc:686] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:11.601624 1388 raft_consensus.cc:504] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.602355 1353 consensus_queue.cc:237] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.604223 1388 leader_election.cc:288] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.606109 1301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.606835 1301 raft_consensus.cc:2454] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.606470 1051 catalog_manager.cc:5495] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 1 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.608099 1192 leader_election.cc:302] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:11.608830 1345 tablet_bootstrap.cc:654] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.609074 1388 raft_consensus.cc:2781] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.609383 1388 raft_consensus.cc:482] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.609695 1388 raft_consensus.cc:3037] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.600111 1302 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:11.606109 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
W20260619 21:15:11.613982 1195 leader_election.cc:341] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.604460 1049 catalog_manager.cc:5495] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 0 to 1, leader changed from <none> to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.621109 1049 catalog_manager.cc:5495] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 0 to 1, leader changed from <none> to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.625378 1345 tablet_bootstrap.cc:492] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.625959 1345 ts_tablet_manager.cc:1397] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.002s
I20260619 21:15:11.628408 1388 raft_consensus.cc:504] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.629182 1345 raft_consensus.cc:348] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.629993 1345 raft_consensus.cc:374] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.630265 1345 raft_consensus.cc:729] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.631114 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.631131 1345 consensus_queue.cc:260] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
W20260619 21:15:11.632680 1195 leader_election.cc:341] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.634016 1345 ts_tablet_manager.cc:1428] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260619 21:15:11.634662 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.635123 1345 tablet_bootstrap.cc:492] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.635236 1302 raft_consensus.cc:3037] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.630364 1343 tablet_bootstrap.cc:654] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.638345 1388 leader_election.cc:288] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.641510 1345 tablet_bootstrap.cc:654] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.644210 1344 tablet_bootstrap.cc:492] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.644776 1344 ts_tablet_manager.cc:1397] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.072s user 0.015s sys 0.012s
I20260619 21:15:11.647141 1302 raft_consensus.cc:2454] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:11.647833 1344 raft_consensus.cc:348] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.648258 1192 leader_election.cc:302] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters: e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:11.649827 1388 raft_consensus.cc:2781] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.650457 1388 raft_consensus.cc:686] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:11.651057 1350 raft_consensus.cc:482] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.651518 1350 raft_consensus.cc:504] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.651584 1388 consensus_queue.cc:237] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.653446 1350 leader_election.cc:288] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:11.655220 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c498a73e8bdb476b975e8119817b81ff" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.655880 1154 raft_consensus.cc:2454] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 0.
I20260619 21:15:11.648588 1344 raft_consensus.cc:374] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.657039 1269 leader_election.cc:302] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.657176 1344 raft_consensus.cc:729] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.658529 1387 raft_consensus.cc:482] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.658643 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c498a73e8bdb476b975e8119817b81ff" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.658245 1344 consensus_queue.cc:260] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.659009 1387 raft_consensus.cc:504] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.660100 1229 raft_consensus.cc:2454] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 0.
I20260619 21:15:11.663522 1387 leader_election.cc:288] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.664515 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.665701 1154 raft_consensus.cc:2454] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.667136 1195 leader_election.cc:302] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:11.663067 1350 raft_consensus.cc:2781] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.668407 1350 raft_consensus.cc:482] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.669113 1387 raft_consensus.cc:2781] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.669548 1387 raft_consensus.cc:482] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.669936 1387 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.668835 1350 raft_consensus.cc:3037] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.674304 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
W20260619 21:15:11.676074 1192 leader_election.cc:341] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.678481 1051 catalog_manager.cc:5495] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 1 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.697885 1344 ts_tablet_manager.cc:1428] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.053s user 0.007s sys 0.003s
I20260619 21:15:11.698027 1387 raft_consensus.cc:504] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.698827 1343 tablet_bootstrap.cc:492] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.698925 1344 tablet_bootstrap.cc:492] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c: Bootstrap starting.
I20260619 21:15:11.699419 1343 ts_tablet_manager.cc:1397] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.132s user 0.023s sys 0.036s
I20260619 21:15:11.701427 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.701524 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.702044 1154 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.702433 1345 tablet_bootstrap.cc:492] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.702180 1350 raft_consensus.cc:504] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.702880 1345 ts_tablet_manager.cc:1397] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.068s user 0.022s sys 0.024s
W20260619 21:15:11.703169 1192 leader_election.cc:341] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:11.703787 1387 leader_election.cc:288] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.705101 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c498a73e8bdb476b975e8119817b81ff" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.705706 1153 raft_consensus.cc:3037] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.705996 1345 raft_consensus.cc:348] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.706670 1345 raft_consensus.cc:374] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.706385 1343 raft_consensus.cc:348] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.706944 1345 raft_consensus.cc:729] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.707052 1343 raft_consensus.cc:374] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.707358 1343 raft_consensus.cc:729] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.707623 1345 consensus_queue.cc:260] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.709703 1345 ts_tablet_manager.cc:1428] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 21:15:11.711540 1350 leader_election.cc:288] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:11.707950 1343 consensus_queue.cc:260] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.712155 1345 tablet_bootstrap.cc:492] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96: Bootstrap starting.
I20260619 21:15:11.714031 1343 ts_tablet_manager.cc:1428] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.014s user 0.007s sys 0.002s
I20260619 21:15:11.714924 1343 tablet_bootstrap.cc:492] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.717281 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c498a73e8bdb476b975e8119817b81ff" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:11.718009 1229 raft_consensus.cc:3037] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.718777 1345 tablet_bootstrap.cc:654] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.720329 1343 tablet_bootstrap.cc:654] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.726590 1229 raft_consensus.cc:2454] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 1.
I20260619 21:15:11.731432 1267 leader_election.cc:302] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:11.732401 1350 raft_consensus.cc:2781] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.738822 1350 raft_consensus.cc:686] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Becoming Leader. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:11.742800 1153 raft_consensus.cc:2454] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 1.
I20260619 21:15:11.740031 1350 consensus_queue.cc:237] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.747107 1154 raft_consensus.cc:2454] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:11.742264 1051 catalog_manager.cc:5495] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 1 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.748234 1195 leader_election.cc:302] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters: 185ab65a08dd4d908675e66d19950d21
I20260619 21:15:11.749223 1399 raft_consensus.cc:2781] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:11.749725 1399 raft_consensus.cc:686] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:11.750711 1399 consensus_queue.cc:237] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.766187 1344 tablet_bootstrap.cc:654] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.773028 1343 tablet_bootstrap.cc:492] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.773473 1343 ts_tablet_manager.cc:1397] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.059s user 0.018s sys 0.039s
I20260619 21:15:11.773801 1345 tablet_bootstrap.cc:492] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96: No bootstrap required, opened a new log
I20260619 21:15:11.774287 1345 ts_tablet_manager.cc:1397] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent bootstrapping tablet: real 0.064s user 0.021s sys 0.028s
I20260619 21:15:11.774185 1050 catalog_manager.cc:5495] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 1 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:11.775544 1344 tablet_bootstrap.cc:492] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c: No bootstrap required, opened a new log
I20260619 21:15:11.776046 1344 ts_tablet_manager.cc:1397] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent bootstrapping tablet: real 0.077s user 0.003s sys 0.009s
I20260619 21:15:11.775949 1343 raft_consensus.cc:348] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.776621 1343 raft_consensus.cc:374] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.776911 1343 raft_consensus.cc:729] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.776889 1345 raft_consensus.cc:348] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.777740 1345 raft_consensus.cc:374] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.778103 1345 raft_consensus.cc:729] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.777751 1343 consensus_queue.cc:260] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.778972 1344 raft_consensus.cc:348] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.779659 1344 raft_consensus.cc:374] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.779944 1344 raft_consensus.cc:729] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.780181 1343 ts_tablet_manager.cc:1428] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260619 21:15:11.780731 1344 consensus_queue.cc:260] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.781029 1345 consensus_queue.cc:260] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.781080 1343 tablet_bootstrap.cc:492] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21: Bootstrap starting.
I20260619 21:15:11.783074 1345 ts_tablet_manager.cc:1428] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260619 21:15:11.787072 1343 tablet_bootstrap.cc:654] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:11.786718 1049 catalog_manager.cc:5495] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 185ab65a08dd4d908675e66d19950d21 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "185ab65a08dd4d908675e66d19950d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:11.788839 1344 ts_tablet_manager.cc:1428] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260619 21:15:11.791718 1343 tablet_bootstrap.cc:492] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21: No bootstrap required, opened a new log
I20260619 21:15:11.792142 1343 ts_tablet_manager.cc:1397] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260619 21:15:11.795329 1343 raft_consensus.cc:348] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.795994 1343 raft_consensus.cc:374] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:11.796255 1343 raft_consensus.cc:729] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Initialized, Role: FOLLOWER
I20260619 21:15:11.796868 1343 consensus_queue.cc:260] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.798978 1343 ts_tablet_manager.cc:1428] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260619 21:15:11.868717 1350 raft_consensus.cc:482] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.869319 1350 raft_consensus.cc:504] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.870985 1350 leader_election.cc:288] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:11.871959 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.872138 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c" is_pre_election: true
I20260619 21:15:11.872673 1154 raft_consensus.cc:2379] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 185ab65a08dd4d908675e66d19950d21 in current term 1: Already voted for candidate 0430d901e2f04acf93e0dfc23551ac1c in this term.
I20260619 21:15:11.873705 1269 leader_election.cc:302] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21; no voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:11.874341 1350 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.878990 1350 raft_consensus.cc:2726] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:11.904594 1348 consensus_queue.cc:1035] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:11.915625 1228 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.919523 1379 consensus_queue.cc:1035] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:11.927289 1350 consensus_queue.cc:1035] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:11.939740 1350 consensus_queue.cc:1035] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:11.948750 1399 raft_consensus.cc:482] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:11.949328 1399 raft_consensus.cc:504] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.949851 1379 consensus_queue.cc:1035] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:11.951354 1399 leader_election.cc:288] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.953663 1301 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.953702 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbda230b058e4a9b8aa6d0dbc5ace417" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21" is_pre_election: true
I20260619 21:15:11.954514 1302 raft_consensus.cc:2454] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.955550 1192 leader_election.cc:302] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:11.956357 1399 raft_consensus.cc:2781] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:11.956698 1399 raft_consensus.cc:482] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:11.956636 1379 consensus_queue.cc:1035] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:11.957156 1399 raft_consensus.cc:3037] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.966207 1379 consensus_queue.cc:1035] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:11.966760 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbda230b058e4a9b8aa6d0dbc5ace417" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" is_pre_election: true
I20260619 21:15:11.967495 1154 raft_consensus.cc:2454] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 0.
I20260619 21:15:11.964767 1399 raft_consensus.cc:504] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:11.974848 1154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbda230b058e4a9b8aa6d0dbc5ace417" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:11.975698 1154 raft_consensus.cc:3037] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.975348 1376 consensus_queue.cc:1035] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:11.982409 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbda230b058e4a9b8aa6d0dbc5ace417" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:11.983209 1303 raft_consensus.cc:3037] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:11.997587 1303 raft_consensus.cc:2454] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:11.974876 1399 leader_election.cc:288] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:11.997594 1154 raft_consensus.cc:2454] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 1.
I20260619 21:15:12.003906 1387 consensus_queue.cc:1035] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.018010 1192 leader_election.cc:302] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:12.018831 1387 raft_consensus.cc:2781] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:12.019443 1387 raft_consensus.cc:686] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:12.020197 1387 consensus_queue.cc:237] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.033303 1399 consensus_queue.cc:1035] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.034658 1050 catalog_manager.cc:5495] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 1 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:12.064654 1399 consensus_queue.cc:1035] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.074928 1153 raft_consensus.cc:3037] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:12.082422 1399 consensus_queue.cc:1035] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.120361 1399 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.131105 1388 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.171149 1399 consensus_queue.cc:1035] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.181151 1399 consensus_queue.cc:1035] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.192144 1350 consensus_queue.cc:1035] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.203305 1364 consensus_queue.cc:1035] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:12.304292 1229 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:57808
I20260619 21:15:12.304968 1229 raft_consensus.cc:595] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:12.319165 1303 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:60232
I20260619 21:15:12.319691 1303 raft_consensus.cc:595] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:12.333981 1229 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:57808
I20260619 21:15:12.334642 1229 raft_consensus.cc:595] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:12.351011 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
from {username='slave'} at 127.0.0.1:39594
I20260619 21:15:12.351541 1153 raft_consensus.cc:595] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Received request to transfer leadership to TS 0430d901e2f04acf93e0dfc23551ac1c
I20260619 21:15:12.437060 1399 consensus_queue.cc:1035] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.447081 1387 consensus_queue.cc:1035] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:12.459143 1412 raft_consensus.cc:979] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:12.459615 1364 raft_consensus.cc:1067] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:12.461126 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39580
I20260619 21:15:12.461803 1153 raft_consensus.cc:482] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:12.462175 1153 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.467309 1153 raft_consensus.cc:504] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.469460 1153 leader_election.cc:288] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:12.469646 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:12.470017 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:12.470242 1229 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.470652 1303 raft_consensus.cc:3032] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:12.470906 1303 raft_consensus.cc:729] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:12.471482 1303 consensus_queue.cc:260] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.472373 1303 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.475111 1229 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:12.476054 1118 leader_election.cc:302] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:12.476723 1376 raft_consensus.cc:2781] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:12.477035 1303 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:12.477118 1376 raft_consensus.cc:686] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:12.478025 1376 consensus_queue.cc:237] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.486510 1050 catalog_manager.cc:5495] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 1 to 2, leader changed from 185ab65a08dd4d908675e66d19950d21 (127.0.99.3) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:12.520337 1348 raft_consensus.cc:979] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 0430d901e2f04acf93e0dfc23551ac1c to start an election
I20260619 21:15:12.520900 1376 raft_consensus.cc:1067] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Signalling peer 0430d901e2f04acf93e0dfc23551ac1c to start an election
I20260619 21:15:12.522405 1229 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
from {username='slave'} at 127.0.0.1:57784
I20260619 21:15:12.522956 1229 raft_consensus.cc:482] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:12.523263 1229 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.528431 1229 raft_consensus.cc:504] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.529990 1229 leader_election.cc:288] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 2 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:12.530858 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:12.531049 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:12.531479 1153 raft_consensus.cc:3032] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:12.531543 1303 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.531774 1153 raft_consensus.cc:729] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:12.532618 1153 consensus_queue.cc:260] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.533624 1153 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.536393 1303 raft_consensus.cc:2454] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 2.
I20260619 21:15:12.537384 1192 leader_election.cc:302] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:12.538282 1387 raft_consensus.cc:2781] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:12.538760 1387 raft_consensus.cc:686] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:12.539163 1153 raft_consensus.cc:2454] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 2.
I20260619 21:15:12.539655 1387 consensus_queue.cc:237] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.546872 1050 catalog_manager.cc:5495] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 1 to 2, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 2 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:12.615993 1399 raft_consensus.cc:979] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:12.616506 1399 raft_consensus.cc:1067] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:12.618009 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39590
I20260619 21:15:12.618587 1153 raft_consensus.cc:482] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:12.618943 1153 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.624984 1153 raft_consensus.cc:504] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.626237 1387 raft_consensus.cc:979] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:12.626631 1399 raft_consensus.cc:1067] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:12.626852 1153 leader_election.cc:288] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:12.627527 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:12.628108 1229 raft_consensus.cc:3032] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:12.628201 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39590
I20260619 21:15:12.628260 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:12.628444 1229 raft_consensus.cc:729] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:12.628819 1303 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.628837 1153 raft_consensus.cc:482] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:12.629513 1153 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.629321 1229 consensus_queue.cc:260] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.630388 1229 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.635401 1229 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:12.635999 1303 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:12.635906 1153 raft_consensus.cc:504] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.636551 1118 leader_election.cc:302] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:12.637668 1376 raft_consensus.cc:2781] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:12.637995 1153 leader_election.cc:288] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:12.638178 1376 raft_consensus.cc:686] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:12.639124 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:12.639089 1376 consensus_queue.cc:237] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.639797 1303 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.640899 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:12.641471 1229 raft_consensus.cc:3032] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:12.641786 1229 raft_consensus.cc:729] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:12.642623 1229 consensus_queue.cc:260] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.643860 1229 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:12.646955 1303 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:12.646664 1050 catalog_manager.cc:5495] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 1 to 2, leader changed from 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:12.647998 1117 leader_election.cc:302] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:12.648828 1376 raft_consensus.cc:2781] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:12.649325 1376 raft_consensus.cc:686] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:12.650120 1376 consensus_queue.cc:237] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:12.650753 1229 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:12.661072 1052 catalog_manager.cc:5495] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 1 to 2, leader changed from 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:12.922290 1153 raft_consensus.cc:1261] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Refusing update from remote peer 0430d901e2f04acf93e0dfc23551ac1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:12.923637 1399 consensus_queue.cc:1035] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:12.930862 1303 raft_consensus.cc:1261] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Refusing update from remote peer 0430d901e2f04acf93e0dfc23551ac1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:12.932133 1435 consensus_queue.cc:1035] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:12.991393 1229 raft_consensus.cc:1261] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:12.992592 1376 consensus_queue.cc:1035] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:13.001013 1303 raft_consensus.cc:1261] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:13.002979 1376 consensus_queue.cc:1035] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:13.105422 1229 raft_consensus.cc:1261] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:13.106698 1437 consensus_queue.cc:1035] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:13.115358 1302 raft_consensus.cc:1261] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:13.117187 1348 consensus_queue.cc:1035] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:13.118835 1229 raft_consensus.cc:1261] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:13.120525 1348 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:13.130007 1302 raft_consensus.cc:1261] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:13.131839 1441 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:15.391211 396 auto_leader_rebalancer-test.cc:339] The leader distribution is
I20260619 21:15:15.392051 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:15.411814 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:33644
I20260619 21:15:15.412616 1153 raft_consensus.cc:595] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Received request to transfer leadership to TS 185ab65a08dd4d908675e66d19950d21
I20260619 21:15:15.415118 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:33644
I20260619 21:15:15.415648 1153 raft_consensus.cc:595] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Received request to transfer leadership to TS 185ab65a08dd4d908675e66d19950d21
I20260619 21:15:15.416709 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260619 21:15:15.417217 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:15.606307 1443 raft_consensus.cc:979] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:15.606930 1449 raft_consensus.cc:1067] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Signalling peer 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:15.608628 1302 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:60212
I20260619 21:15:15.609220 1302 raft_consensus.cc:482] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:15.609725 1302 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:15.617915 1302 raft_consensus.cc:504] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:15.619748 1302 leader_election.cc:288] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 2 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:15.620621 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:15.621047 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:15.621448 1153 raft_consensus.cc:3032] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:15.621685 1228 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:15.621858 1153 raft_consensus.cc:729] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:15.622745 1153 consensus_queue.cc:260] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:15.624207 1153 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:15.628082 1228 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 2.
I20260619 21:15:15.629390 1267 leader_election.cc:302] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:15.630237 1350 raft_consensus.cc:2781] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:15.630286 1153 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 2.
I20260619 21:15:15.631940 1350 raft_consensus.cc:686] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Becoming Leader. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:15.632958 1350 consensus_queue.cc:237] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:15.641075 1052 catalog_manager.cc:5495] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 reported cstate change: term changed from 1 to 2, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 185ab65a08dd4d908675e66d19950d21 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "185ab65a08dd4d908675e66d19950d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:15.811411 1443 raft_consensus.cc:979] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:15.811928 1449 raft_consensus.cc:1067] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Signalling peer 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:15.813349 1302 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:60212
I20260619 21:15:15.813838 1302 raft_consensus.cc:482] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:15.814110 1302 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:15.820048 1302 raft_consensus.cc:504] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:15.821790 1302 leader_election.cc:288] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 3 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:15.822746 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:15.822835 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:15.823509 1153 raft_consensus.cc:3032] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:15.823575 1228 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:15.824571 1153 raft_consensus.cc:729] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:15.825428 1153 consensus_queue.cc:260] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:15.826579 1153 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:15.829321 1228 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 3.
I20260619 21:15:15.830315 1267 leader_election.cc:302] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:15.831111 1350 raft_consensus.cc:2781] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:15.831575 1350 raft_consensus.cc:686] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Becoming Leader. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:15.832504 1350 consensus_queue.cc:237] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:15.832994 1153 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 3.
I20260619 21:15:15.844753 1052 catalog_manager.cc:5495] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 reported cstate change: term changed from 2 to 3, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 185ab65a08dd4d908675e66d19950d21 (127.0.99.3). New cstate: current_term: 3 leader_uuid: "185ab65a08dd4d908675e66d19950d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:16.054447 1228 raft_consensus.cc:1261] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:16.055764 1350 consensus_queue.cc:1035] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:16.066708 1153 raft_consensus.cc:1261] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:16.071573 1450 consensus_queue.cc:1035] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:16.329629 1153 raft_consensus.cc:1261] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:16.337128 1450 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260619 21:15:16.348476 1229 raft_consensus.cc:1261] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:16.354619 1464 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:15:16.418099 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:16.423830 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:16.424355 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:17.425354 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:17.430671 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:17.431142 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:18.432089 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:18.438757 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:18.439204 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:19.440160 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:19.445478 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:19.445950 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:20.446712 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:20.452087 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:20.452478 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:21.453249 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:21.458760 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:21.459149 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:22.459964 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:22.465515 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:22.465955 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:23.466732 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:23.471871 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:23.472265 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:24.473165 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:24.479859 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:24.480453 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:25.513128 396 auto_leader_rebalancer-test.cc:358] The leader distribution is
I20260619 21:15:25.518508 1302 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:60232
I20260619 21:15:25.519274 1302 raft_consensus.cc:595] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:25.532811 1302 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:60232
I20260619 21:15:25.533443 1302 raft_consensus.cc:595] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:25.538178 1228 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:57808
I20260619 21:15:25.538748 1228 raft_consensus.cc:595] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:25.543534 1228 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:57808
I20260619 21:15:25.544000 1228 raft_consensus.cc:595] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:25.548542 1228 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5e0108caeb6467698431893252bb550"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
mode: GRACEFUL
new_leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:57808
I20260619 21:15:25.549037 1228 raft_consensus.cc:595] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Received request to transfer leadership to TS e98ed1b6ac984ca0bfed51bac7695a96
I20260619 21:15:25.690362 1451 raft_consensus.cc:979] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.691105 1471 raft_consensus.cc:1067] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.692800 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39580
I20260619 21:15:25.693316 1153 raft_consensus.cc:482] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:25.693655 1153 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:25.700074 1153 raft_consensus.cc:504] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.701793 1153 leader_election.cc:288] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 3 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:25.702919 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:25.702970 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e625e8cd81b24cdd8f9ee28de8e0ee9c" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:25.703728 1228 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:25.703824 1302 raft_consensus.cc:3032] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:25.704174 1302 raft_consensus.cc:729] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:25.704852 1302 consensus_queue.cc:260] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.705986 1302 raft_consensus.cc:3037] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:25.709424 1228 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 3.
I20260619 21:15:25.710726 1118 leader_election.cc:302] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:25.711620 1302 raft_consensus.cc:2454] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 3.
I20260619 21:15:25.711748 1468 raft_consensus.cc:2781] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:25.713304 1468 raft_consensus.cc:686] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:25.714210 1468 consensus_queue.cc:237] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.722218 1052 catalog_manager.cc:5495] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 2 to 3, leader changed from 185ab65a08dd4d908675e66d19950d21 (127.0.99.3) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 3 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:25.723308 1469 raft_consensus.cc:979] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.723953 1474 raft_consensus.cc:1067] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.725840 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39590
I20260619 21:15:25.726356 1153 raft_consensus.cc:482] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:25.726642 1153 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:25.737416 1153 raft_consensus.cc:504] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.741158 1153 leader_election.cc:288] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 3 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:25.742674 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:25.742801 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f86dbcdc0853415d8bd72bf06f596175" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:25.744566 1228 raft_consensus.cc:3032] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:25.744566 1302 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:25.745703 1228 raft_consensus.cc:729] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:25.748265 1228 consensus_queue.cc:260] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.752737 1228 raft_consensus.cc:3037] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:25.761536 1302 raft_consensus.cc:2454] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 3.
I20260619 21:15:25.762609 1228 raft_consensus.cc:2454] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 3.
I20260619 21:15:25.764025 1117 leader_election.cc:302] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:25.766505 1468 raft_consensus.cc:2781] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:25.767848 1468 raft_consensus.cc:686] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:25.770128 1468 consensus_queue.cc:237] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.777366 1052 catalog_manager.cc:5495] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 2 to 3, leader changed from 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 3 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:25.780320 1469 raft_consensus.cc:979] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.780790 1474 raft_consensus.cc:1067] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.782346 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39590
I20260619 21:15:25.783732 1153 raft_consensus.cc:482] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:25.784359 1153 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:25.793402 1153 raft_consensus.cc:504] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.794907 1153 leader_election.cc:288] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:25.796027 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:25.795907 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:25.796593 1302 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:25.796697 1229 raft_consensus.cc:3032] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:25.797138 1229 raft_consensus.cc:729] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:25.797885 1229 consensus_queue.cc:260] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.799129 1229 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:25.802639 1302 raft_consensus.cc:2454] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:25.803494 1117 leader_election.cc:302] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:25.804206 1468 raft_consensus.cc:2781] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:25.804662 1468 raft_consensus.cc:686] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:25.805529 1468 consensus_queue.cc:237] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.806187 1229 raft_consensus.cc:2454] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:25.813373 1052 catalog_manager.cc:5495] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 1 to 2, leader changed from 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:25.852149 1469 raft_consensus.cc:979] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.852629 1469 raft_consensus.cc:1067] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.854113 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5e0108caeb6467698431893252bb550"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39590
I20260619 21:15:25.854610 1153 raft_consensus.cc:482] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:25.854875 1153 raft_consensus.cc:3037] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:25.859539 1153 raft_consensus.cc:504] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.861101 1153 leader_election.cc:288] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:25.862031 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:25.862305 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e0108caeb6467698431893252bb550" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:25.862638 1229 raft_consensus.cc:3032] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:25.862843 1302 raft_consensus.cc:3037] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:25.863029 1229 raft_consensus.cc:729] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:25.863881 1229 consensus_queue.cc:260] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.865072 1229 raft_consensus.cc:3037] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:25.867981 1302 raft_consensus.cc:2454] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:25.869081 1117 leader_election.cc:302] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ab65a08dd4d908675e66d19950d21, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:25.869912 1468 raft_consensus.cc:2781] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:25.870334 1468 raft_consensus.cc:686] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:25.870617 1229 raft_consensus.cc:2454] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 2.
I20260619 21:15:25.871176 1468 consensus_queue.cc:237] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.879033 1052 catalog_manager.cc:5495] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 1 to 2, leader changed from 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:25.922286 1451 raft_consensus.cc:979] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: : Instructing follower e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.922724 1471 raft_consensus.cc:1067] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Signalling peer e98ed1b6ac984ca0bfed51bac7695a96 to start an election
I20260619 21:15:25.924253 1153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
from {username='slave'} at 127.0.0.1:39580
I20260619 21:15:25.924686 1153 raft_consensus.cc:482] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:25.925096 1153 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:25.930320 1153 raft_consensus.cc:504] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.931945 1153 leader_election.cc:288] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 4 election: Requested vote from peers 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:25.932928 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:25.933224 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b134eb5e43f4a43ab5bad911099ec60" candidate_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:25.933565 1229 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:25.933763 1302 raft_consensus.cc:3032] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:15:25.934084 1302 raft_consensus.cc:729] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:25.934793 1302 consensus_queue.cc:260] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.935738 1302 raft_consensus.cc:3037] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:25.939841 1229 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 4.
I20260619 21:15:25.940889 1118 leader_election.cc:302] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, e98ed1b6ac984ca0bfed51bac7695a96; no voters:
I20260619 21:15:25.941715 1468 raft_consensus.cc:2781] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:15:25.941951 1302 raft_consensus.cc:2454] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e98ed1b6ac984ca0bfed51bac7695a96 in term 4.
I20260619 21:15:25.942265 1468 raft_consensus.cc:686] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 4 LEADER]: Becoming Leader. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:25.943516 1468 consensus_queue.cc:237] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:25.953549 1052 catalog_manager.cc:5495] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 reported cstate change: term changed from 3 to 4, leader changed from 185ab65a08dd4d908675e66d19950d21 (127.0.99.3) to e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1). New cstate: current_term: 4 leader_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:26.145332 1302 raft_consensus.cc:1261] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:26.146553 1468 consensus_queue.cc:1035] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:26.156629 1229 raft_consensus.cc:1261] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:26.159541 1472 consensus_queue.cc:1035] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:15:26.271955 1229 raft_consensus.cc:1261] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:26.273342 1472 consensus_queue.cc:1035] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:26.274488 1229 raft_consensus.cc:1261] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:26.275697 1472 consensus_queue.cc:1035] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:26.283766 1302 raft_consensus.cc:1261] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:26.289565 1302 raft_consensus.cc:1261] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:26.292349 1492 consensus_queue.cc:1035] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:26.295078 1491 consensus_queue.cc:1035] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260619 21:15:26.309739 1301 raft_consensus.cc:1261] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:26.313115 1472 consensus_queue.cc:1035] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:26.331270 1229 raft_consensus.cc:1261] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:26.332595 1472 consensus_queue.cc:1035] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:26.450464 1303 raft_consensus.cc:1261] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 4 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:15:26.451840 1472 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:15:26.457768 1229 raft_consensus.cc:1261] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 4 FOLLOWER]: Refusing update from remote peer e98ed1b6ac984ca0bfed51bac7695a96: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:15:26.459247 1493 consensus_queue.cc:1035] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:15:28.584308 396 auto_leader_rebalancer-test.cc:373] The leader distribution is
I20260619 21:15:28.585098 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:28.591742 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:33644
I20260619 21:15:28.592252 1153 raft_consensus.cc:595] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Received request to transfer leadership to TS 185ab65a08dd4d908675e66d19950d21
I20260619 21:15:28.594477 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:33644
I20260619 21:15:28.594926 1153 raft_consensus.cc:595] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Received request to transfer leadership to TS 185ab65a08dd4d908675e66d19950d21
I20260619 21:15:28.596817 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
from {username='slave'} at 127.0.0.1:33644
I20260619 21:15:28.597260 1153 raft_consensus.cc:595] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Received request to transfer leadership to TS 0430d901e2f04acf93e0dfc23551ac1c
I20260619 21:15:28.599274 1153 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567"
dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
mode: GRACEFUL
new_leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
from {username='slave'} at 127.0.0.1:33644
I20260619 21:15:28.599673 1153 raft_consensus.cc:595] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Received request to transfer leadership to TS 0430d901e2f04acf93e0dfc23551ac1c
I20260619 21:15:28.600575 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260619 21:15:28.601121 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:28.646879 1472 raft_consensus.cc:979] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:28.647387 1500 raft_consensus.cc:1067] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Signalling peer 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:28.648833 1303 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:60212
I20260619 21:15:28.649324 1303 raft_consensus.cc:482] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:28.649644 1303 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.655026 1303 raft_consensus.cc:504] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.657101 1303 leader_election.cc:288] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 3 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:28.658156 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:28.658046 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367501a407f64ec89b6080d0c2277fea" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:28.658668 1228 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.658721 1153 raft_consensus.cc:3032] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:28.659101 1153 raft_consensus.cc:729] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:28.659749 1153 consensus_queue.cc:260] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.660638 1153 raft_consensus.cc:3037] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.663563 1228 raft_consensus.cc:2454] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 3.
I20260619 21:15:28.664634 1267 leader_election.cc:302] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:28.665073 1153 raft_consensus.cc:2454] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 3.
I20260619 21:15:28.665408 1471 raft_consensus.cc:2781] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:28.666548 1471 raft_consensus.cc:686] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Becoming Leader. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:28.667399 1471 consensus_queue.cc:237] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.673887 1052 catalog_manager.cc:5495] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 reported cstate change: term changed from 2 to 3, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 185ab65a08dd4d908675e66d19950d21 (127.0.99.3). New cstate: current_term: 3 leader_uuid: "185ab65a08dd4d908675e66d19950d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:28.837950 1472 raft_consensus.cc:979] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:28.838496 1500 raft_consensus.cc:1067] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Signalling peer 185ab65a08dd4d908675e66d19950d21 to start an election
I20260619 21:15:28.839941 1303 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b"
dest_uuid: "185ab65a08dd4d908675e66d19950d21"
from {username='slave'} at 127.0.0.1:60212
I20260619 21:15:28.840471 1303 raft_consensus.cc:482] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:28.840780 1303 raft_consensus.cc:3037] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:28.845445 1303 raft_consensus.cc:504] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.846915 1303 leader_election.cc:288] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 2 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879)
I20260619 21:15:28.847802 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:28.847882 1228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370269df5d5d434b8e3ff6a330cafe3b" candidate_uuid: "185ab65a08dd4d908675e66d19950d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
I20260619 21:15:28.848479 1153 raft_consensus.cc:3032] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:28.848547 1228 raft_consensus.cc:3037] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:28.848875 1153 raft_consensus.cc:729] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:28.849705 1153 consensus_queue.cc:260] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.850932 1153 raft_consensus.cc:3037] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:28.854065 1228 raft_consensus.cc:2454] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 2.
I20260619 21:15:28.854913 1267 leader_election.cc:302] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:28.855603 1153 raft_consensus.cc:2454] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 185ab65a08dd4d908675e66d19950d21 in term 2.
I20260619 21:15:28.855623 1471 raft_consensus.cc:2781] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:28.856288 1471 raft_consensus.cc:686] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Becoming Leader. State: Replica: 185ab65a08dd4d908675e66d19950d21, State: Running, Role: LEADER
I20260619 21:15:28.857127 1471 consensus_queue.cc:237] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.863425 1052 catalog_manager.cc:5495] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 reported cstate change: term changed from 1 to 2, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 185ab65a08dd4d908675e66d19950d21 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "185ab65a08dd4d908675e66d19950d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:28.922763 1472 raft_consensus.cc:979] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 0430d901e2f04acf93e0dfc23551ac1c to start an election
I20260619 21:15:28.923247 1500 raft_consensus.cc:1067] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Signalling peer 0430d901e2f04acf93e0dfc23551ac1c to start an election
I20260619 21:15:28.924672 1228 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
from {username='slave'} at 127.0.0.1:57784
I20260619 21:15:28.925242 1228 raft_consensus.cc:482] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:28.925522 1228 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.930379 1228 raft_consensus.cc:504] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.932626 1228 leader_election.cc:288] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 3 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:28.933785 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:28.933794 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7905cb353dda4e818a29a34f9f6bac0c" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:28.934548 1303 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.934680 1153 raft_consensus.cc:3032] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:28.934952 1153 raft_consensus.cc:729] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:28.935447 1153 consensus_queue.cc:260] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.936316 1153 raft_consensus.cc:3037] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.940285 1303 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 3.
I20260619 21:15:28.941457 1192 leader_election.cc:302] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:28.942180 1469 raft_consensus.cc:2781] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:28.943012 1153 raft_consensus.cc:2454] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 3.
I20260619 21:15:28.943436 1469 raft_consensus.cc:686] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:28.944406 1469 consensus_queue.cc:237] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.951746 1052 catalog_manager.cc:5495] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 2 to 3, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 3 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:28.974514 1472 raft_consensus.cc:979] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: : Instructing follower 0430d901e2f04acf93e0dfc23551ac1c to start an election
I20260619 21:15:28.974936 1500 raft_consensus.cc:1067] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Signalling peer 0430d901e2f04acf93e0dfc23551ac1c to start an election
I20260619 21:15:28.976342 1228 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567"
dest_uuid: "0430d901e2f04acf93e0dfc23551ac1c"
from {username='slave'} at 127.0.0.1:57784
I20260619 21:15:28.976876 1228 raft_consensus.cc:482] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 21:15:28.977262 1228 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.982074 1228 raft_consensus.cc:504] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.983536 1228 leader_election.cc:288] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 3 election: Requested vote from peers e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165), 185ab65a08dd4d908675e66d19950d21 (127.0.99.3:44547)
I20260619 21:15:28.984462 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e98ed1b6ac984ca0bfed51bac7695a96"
I20260619 21:15:28.984472 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5976ef9d5574abcb8ad2f3eb80e2567" candidate_uuid: "0430d901e2f04acf93e0dfc23551ac1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "185ab65a08dd4d908675e66d19950d21"
I20260619 21:15:28.985070 1153 raft_consensus.cc:3032] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:28.985136 1303 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.985356 1153 raft_consensus.cc:729] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e98ed1b6ac984ca0bfed51bac7695a96, State: Running, Role: LEADER
I20260619 21:15:28.986019 1153 consensus_queue.cc:260] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:28.986979 1153 raft_consensus.cc:3037] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:28.990468 1303 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 3.
I20260619 21:15:28.991505 1153 raft_consensus.cc:2454] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0430d901e2f04acf93e0dfc23551ac1c in term 3.
I20260619 21:15:28.991691 1192 leader_election.cc:302] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0430d901e2f04acf93e0dfc23551ac1c, 185ab65a08dd4d908675e66d19950d21; no voters:
I20260619 21:15:28.992461 1469 raft_consensus.cc:2781] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:28.992889 1469 raft_consensus.cc:686] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 LEADER]: Becoming Leader. State: Replica: 0430d901e2f04acf93e0dfc23551ac1c, State: Running, Role: LEADER
I20260619 21:15:28.993703 1469 consensus_queue.cc:237] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } }
I20260619 21:15:29.001227 1052 catalog_manager.cc:5495] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c reported cstate change: term changed from 2 to 3, leader changed from e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1) to 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2). New cstate: current_term: 3 leader_uuid: "0430d901e2f04acf93e0dfc23551ac1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:29.107527 1153 raft_consensus.cc:1261] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:29.108920 1471 consensus_queue.cc:1035] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:29.118659 1228 raft_consensus.cc:1261] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:29.119951 1471 consensus_queue.cc:1035] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:29.255901 1228 raft_consensus.cc:1261] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:29.257436 1471 consensus_queue.cc:1035] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0430d901e2f04acf93e0dfc23551ac1c" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 35879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:29.267045 1153 raft_consensus.cc:1261] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Refusing update from remote peer 185ab65a08dd4d908675e66d19950d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:29.268918 1471 consensus_queue.cc:1035] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:29.413795 1303 raft_consensus.cc:1261] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Refusing update from remote peer 0430d901e2f04acf93e0dfc23551ac1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:29.414986 1469 consensus_queue.cc:1035] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:29.426548 1153 raft_consensus.cc:1261] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Refusing update from remote peer 0430d901e2f04acf93e0dfc23551ac1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:29.428013 1520 consensus_queue.cc:1035] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:15:29.473342 1153 raft_consensus.cc:1261] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Refusing update from remote peer 0430d901e2f04acf93e0dfc23551ac1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:29.474519 1520 consensus_queue.cc:1035] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e98ed1b6ac984ca0bfed51bac7695a96" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:29.483032 1303 raft_consensus.cc:1261] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Refusing update from remote peer 0430d901e2f04acf93e0dfc23551ac1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:15:29.484983 1508 consensus_queue.cc:1035] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ab65a08dd4d908675e66d19950d21" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:15:29.602051 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:29.607366 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:29.607829 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:30.608825 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:30.615021 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:30.615598 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:31.616640 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:31.623926 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:31.624743 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:32.625842 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:32.633493 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:32.634013 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:33.634914 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:33.641893 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:33.642432 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:34.670672 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:34.681391 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:34.681908 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:35.683179 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:35.689428 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:35.690030 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260619 21:15:35.719709 1175 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac3a0 after lost signal to thread 399
W20260619 21:15:35.720840 1175 debug-util.cc:398] Leaking SignalData structure 0x7b08001e2b80 after lost signal to thread 1086
W20260619 21:15:35.722072 1175 debug-util.cc:398] Leaking SignalData structure 0x7b0800057b80 after lost signal to thread 1178
W20260619 21:15:35.723248 1175 debug-util.cc:398] Leaking SignalData structure 0x7b08000bde60 after lost signal to thread 1253
W20260619 21:15:35.724383 1175 debug-util.cc:398] Leaking SignalData structure 0x7b08001544c0 after lost signal to thread 1327
I20260619 21:15:36.690945 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:36.696321 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:36.696794 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260619 21:15:37.790383 396 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260619 21:15:37.795818 396 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260619 21:15:37.796291 396 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260619 21:15:38.210316 1324 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8140 after lost signal to thread 399
W20260619 21:15:38.211215 1324 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8820 after lost signal to thread 1086
W20260619 21:15:38.212209 1324 debug-util.cc:398] Leaking SignalData structure 0x7b08000a8e00 after lost signal to thread 1178
W20260619 21:15:38.213267 1324 debug-util.cc:398] Leaking SignalData structure 0x7b08000b19c0 after lost signal to thread 1253
W20260619 21:15:38.214112 1324 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5500 after lost signal to thread 1327
I20260619 21:15:38.833837 396 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20260619 21:15:38.834496 396 tablet_server.cc:178] TabletServer@127.0.99.1:0 shutting down...
W20260619 21:15:38.863555 1269 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260619 21:15:38.871608 1269 consensus_peers.cc:487] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 -> Peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Couldn't send request to peer e98ed1b6ac984ca0bfed51bac7695a96. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:38.874799 396 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260619 21:15:38.875797 396 tablet_replica.cc:330] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.876598 396 raft_consensus.cc:2229] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 LEADER]: Raft consensus shutting down.
I20260619 21:15:38.878057 396 raft_consensus.cc:2258] T b5e0108caeb6467698431893252bb550 P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.880273 396 tablet_replica.cc:330] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.880923 396 raft_consensus.cc:2229] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:38.881523 396 raft_consensus.cc:2258] T fbda230b058e4a9b8aa6d0dbc5ace417 P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.884104 396 tablet_replica.cc:330] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.884619 396 raft_consensus.cc:2229] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:38.885632 396 raft_consensus.cc:2258] T f86dbcdc0853415d8bd72bf06f596175 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.887698 396 tablet_replica.cc:330] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.888221 396 raft_consensus.cc:2229] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:38.888648 396 raft_consensus.cc:2258] T c498a73e8bdb476b975e8119817b81ff P e98ed1b6ac984ca0bfed51bac7695a96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.890538 396 tablet_replica.cc:330] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.891055 396 raft_consensus.cc:2229] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:38.891527 396 raft_consensus.cc:2258] T 7905cb353dda4e818a29a34f9f6bac0c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.893515 396 tablet_replica.cc:330] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.894018 396 raft_consensus.cc:2229] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:38.894510 396 raft_consensus.cc:2258] T 367501a407f64ec89b6080d0c2277fea P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.896409 396 tablet_replica.cc:330] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.896917 396 raft_consensus.cc:2229] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 4 LEADER]: Raft consensus shutting down.
I20260619 21:15:38.897889 396 raft_consensus.cc:2258] T 0b134eb5e43f4a43ab5bad911099ec60 P e98ed1b6ac984ca0bfed51bac7695a96 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.899978 396 tablet_replica.cc:330] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.900441 396 raft_consensus.cc:2229] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:38.901362 396 raft_consensus.cc:2258] T e625e8cd81b24cdd8f9ee28de8e0ee9c P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.903462 396 tablet_replica.cc:330] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.903973 396 raft_consensus.cc:2229] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:38.904471 396 raft_consensus.cc:2258] T b5976ef9d5574abcb8ad2f3eb80e2567 P e98ed1b6ac984ca0bfed51bac7695a96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.906448 396 tablet_replica.cc:330] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96: stopping tablet replica
I20260619 21:15:38.906936 396 raft_consensus.cc:2229] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:38.907418 396 raft_consensus.cc:2258] T 370269df5d5d434b8e3ff6a330cafe3b P e98ed1b6ac984ca0bfed51bac7695a96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:38.958580 396 tablet_server.cc:194] TabletServer@127.0.99.1:0 shutdown complete.
I20260619 21:15:38.984131 396 tablet_server.cc:178] TabletServer@127.0.99.2:0 shutting down...
W20260619 21:15:39.011016 1195 consensus_peers.cc:487] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c -> Peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Couldn't send request to peer e98ed1b6ac984ca0bfed51bac7695a96. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.1:36165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260619 21:15:39.011111 1267 consensus_peers.cc:487] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 -> Peer 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879): Couldn't send request to peer 0430d901e2f04acf93e0dfc23551ac1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260619 21:15:39.019884 1195 consensus_peers.cc:487] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c -> Peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Couldn't send request to peer e98ed1b6ac984ca0bfed51bac7695a96. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.1:36165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:39.035616 396 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260619 21:15:39.036413 396 tablet_replica.cc:330] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.037253 396 raft_consensus.cc:2229] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 LEADER]: Raft consensus shutting down.
I20260619 21:15:39.038280 396 raft_consensus.cc:2258] T fbda230b058e4a9b8aa6d0dbc5ace417 P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.040405 396 tablet_replica.cc:330] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.040993 396 raft_consensus.cc:2229] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.041472 396 raft_consensus.cc:2258] T b5e0108caeb6467698431893252bb550 P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.043515 396 tablet_replica.cc:330] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.044036 396 raft_consensus.cc:2229] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.044512 396 raft_consensus.cc:2258] T f86dbcdc0853415d8bd72bf06f596175 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.046329 396 tablet_replica.cc:330] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.046872 396 raft_consensus.cc:2229] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:39.047793 396 raft_consensus.cc:2258] T b5976ef9d5574abcb8ad2f3eb80e2567 P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.050325 396 tablet_replica.cc:330] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.050978 396 raft_consensus.cc:2229] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.051672 396 raft_consensus.cc:2258] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260619 21:15:39.052351 1267 consensus_peers.cc:487] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 -> Peer 0430d901e2f04acf93e0dfc23551ac1c (127.0.99.2:35879): Couldn't send request to peer 0430d901e2f04acf93e0dfc23551ac1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:39.053797 396 tablet_replica.cc:330] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.054347 396 raft_consensus.cc:2229] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.054752 396 raft_consensus.cc:2258] T c498a73e8bdb476b975e8119817b81ff P 0430d901e2f04acf93e0dfc23551ac1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.056574 396 tablet_replica.cc:330] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.057112 396 raft_consensus.cc:2229] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:39.057955 396 raft_consensus.cc:2258] T 7905cb353dda4e818a29a34f9f6bac0c P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.059690 396 tablet_replica.cc:330] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.060168 396 raft_consensus.cc:2229] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.060566 396 raft_consensus.cc:2258] T 367501a407f64ec89b6080d0c2277fea P 0430d901e2f04acf93e0dfc23551ac1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.062350 396 tablet_replica.cc:330] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.062822 396 raft_consensus.cc:2229] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.063325 396 raft_consensus.cc:2258] T 0b134eb5e43f4a43ab5bad911099ec60 P 0430d901e2f04acf93e0dfc23551ac1c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.065055 396 tablet_replica.cc:330] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c: stopping tablet replica
I20260619 21:15:39.065464 396 raft_consensus.cc:2229] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.065891 396 raft_consensus.cc:2258] T 370269df5d5d434b8e3ff6a330cafe3b P 0430d901e2f04acf93e0dfc23551ac1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260619 21:15:39.087375 1269 consensus_peers.cc:487] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 -> Peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Couldn't send request to peer e98ed1b6ac984ca0bfed51bac7695a96. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.1:36165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:39.091372 396 tablet_server.cc:194] TabletServer@127.0.99.2:0 shutdown complete.
W20260619 21:15:39.097774 1269 consensus_peers.cc:487] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 -> Peer e98ed1b6ac984ca0bfed51bac7695a96 (127.0.99.1:36165): Couldn't send request to peer e98ed1b6ac984ca0bfed51bac7695a96. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.1:36165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:39.116814 396 tablet_server.cc:178] TabletServer@127.0.99.3:0 shutting down...
I20260619 21:15:39.149843 396 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260619 21:15:39.150691 396 tablet_replica.cc:330] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.151651 396 raft_consensus.cc:2229] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.152261 396 raft_consensus.cc:2258] T fbda230b058e4a9b8aa6d0dbc5ace417 P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.155246 396 tablet_replica.cc:330] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.156016 396 raft_consensus.cc:2229] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.156625 396 raft_consensus.cc:2258] T f86dbcdc0853415d8bd72bf06f596175 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.159189 396 tablet_replica.cc:330] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.159911 396 raft_consensus.cc:2229] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.160473 396 raft_consensus.cc:2258] T b5e0108caeb6467698431893252bb550 P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.163285 396 tablet_replica.cc:330] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.164049 396 raft_consensus.cc:2229] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.164623 396 raft_consensus.cc:2258] T e625e8cd81b24cdd8f9ee28de8e0ee9c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.167035 396 tablet_replica.cc:330] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.167750 396 raft_consensus.cc:2229] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.168320 396 raft_consensus.cc:2258] T b5976ef9d5574abcb8ad2f3eb80e2567 P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.170800 396 tablet_replica.cc:330] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.171588 396 raft_consensus.cc:2229] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:15:39.172806 396 raft_consensus.cc:2258] T 367501a407f64ec89b6080d0c2277fea P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.175431 396 tablet_replica.cc:330] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.176177 396 raft_consensus.cc:2229] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.176846 396 raft_consensus.cc:2258] T 0b134eb5e43f4a43ab5bad911099ec60 P 185ab65a08dd4d908675e66d19950d21 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.179280 396 tablet_replica.cc:330] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.179976 396 raft_consensus.cc:2229] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 1 LEADER]: Raft consensus shutting down.
I20260619 21:15:39.181066 396 raft_consensus.cc:2258] T c498a73e8bdb476b975e8119817b81ff P 185ab65a08dd4d908675e66d19950d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.183439 396 tablet_replica.cc:330] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.184080 396 raft_consensus.cc:2229] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:15:39.184645 396 raft_consensus.cc:2258] T 7905cb353dda4e818a29a34f9f6bac0c P 185ab65a08dd4d908675e66d19950d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:39.187064 396 tablet_replica.cc:330] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21: stopping tablet replica
I20260619 21:15:39.187671 396 raft_consensus.cc:2229] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 2 LEADER]: Raft consensus shutting down.
I20260619 21:15:39.188719 396 raft_consensus.cc:2258] T 370269df5d5d434b8e3ff6a330cafe3b P 185ab65a08dd4d908675e66d19950d21 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260619 21:15:40.224049 396 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 1324)
I20260619 21:15:40.745297 396 tablet_server.cc:194] TabletServer@127.0.99.3:0 shutdown complete.
I20260619 21:15:40.766250 396 master.cc:537] Master@127.0.99.62:45773 shutting down...
I20260619 21:15:40.790352 396 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 1 LEADER]: Raft consensus shutting down.
I20260619 21:15:40.791038 396 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 21:15:40.791404 396 tablet_replica.cc:330] T 00000000000000000000000000000000 P 59415aedeaeb4d75ba16aa3d1bc61903: stopping tablet replica
I20260619 21:15:40.811796 396 master.cc:552] Master@127.0.99.62:45773 shutdown complete.
0430d901e2f04acf93e0dfc23551ac1c 4
185ab65a08dd4d908675e66d19950d21 1
e98ed1b6ac984ca0bfed51bac7695a96 5
0430d901e2f04acf93e0dfc23551ac1c 4
185ab65a08dd4d908675e66d19950d21 3
e98ed1b6ac984ca0bfed51bac7695a96 3
0430d901e2f04acf93e0dfc23551ac1c 1
185ab65a08dd4d908675e66d19950d21 1
e98ed1b6ac984ca0bfed51bac7695a96 8
0430d901e2f04acf93e0dfc23551ac1c 3
185ab65a08dd4d908675e66d19950d21 3
e98ed1b6ac984ca0bfed51bac7695a96 4
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (30672 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260619 21:15:40.858949 396 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.62:42975
I20260619 21:15:40.860075 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:40.866827 1543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:40.867141 1542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:40.867921 1545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:40.869043 396 server_base.cc:1034] running on GCE node
I20260619 21:15:40.870136 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:40.870328 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:40.870508 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903740870494 us; error 0 us; skew 500 ppm
I20260619 21:15:40.871098 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:40.879122 396 webserver.cc:460] Webserver started at http://127.0.99.62:46313/ using document root <none> and password file <none>
I20260619 21:15:40.879709 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:40.879913 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:40.880280 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:40.881690 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/master-0-root/instance:
uuid: "2027366fd3a2418280b4f598c8498021"
format_stamp: "Formatted at 2026-06-19 21:15:40 on dist-test-slave-nlqf"
I20260619 21:15:40.886507 396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 21:15:40.890249 1550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:40.891276 396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 21:15:40.891557 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/master-0-root
uuid: "2027366fd3a2418280b4f598c8498021"
format_stamp: "Formatted at 2026-06-19 21:15:40 on dist-test-slave-nlqf"
I20260619 21:15:40.891815 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:40.914753 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:40.916198 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:40.959944 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.62:42975
I20260619 21:15:40.960075 1601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.62:42975 every 8 connection(s)
I20260619 21:15:40.964421 1602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:40.975382 1602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Bootstrap starting.
I20260619 21:15:40.980111 1602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:40.984797 1602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: No bootstrap required, opened a new log
I20260619 21:15:40.987072 1602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2027366fd3a2418280b4f598c8498021" member_type: VOTER }
I20260619 21:15:40.987609 1602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:40.987886 1602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2027366fd3a2418280b4f598c8498021, State: Initialized, Role: FOLLOWER
I20260619 21:15:40.988504 1602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2027366fd3a2418280b4f598c8498021" member_type: VOTER }
I20260619 21:15:40.988982 1602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 21:15:40.989233 1602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 21:15:40.989523 1602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:40.994450 1602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2027366fd3a2418280b4f598c8498021" member_type: VOTER }
I20260619 21:15:40.995007 1602 leader_election.cc:302] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2027366fd3a2418280b4f598c8498021; no voters:
I20260619 21:15:40.996250 1602 leader_election.cc:288] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 21:15:40.996687 1605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:40.998219 1605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [term 1 LEADER]: Becoming Leader. State: Replica: 2027366fd3a2418280b4f598c8498021, State: Running, Role: LEADER
I20260619 21:15:40.998896 1605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2027366fd3a2418280b4f598c8498021" member_type: VOTER }
I20260619 21:15:40.999447 1602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 21:15:41.003753 1606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2027366fd3a2418280b4f598c8498021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2027366fd3a2418280b4f598c8498021" member_type: VOTER } }
I20260619 21:15:41.004597 1606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [sys.catalog]: This master's current role is: LEADER
I20260619 21:15:41.004361 1607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2027366fd3a2418280b4f598c8498021. Latest consensus state: current_term: 1 leader_uuid: "2027366fd3a2418280b4f598c8498021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2027366fd3a2418280b4f598c8498021" member_type: VOTER } }
I20260619 21:15:41.005789 1607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021 [sys.catalog]: This master's current role is: LEADER
I20260619 21:15:41.007522 1613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 21:15:41.015517 1613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 21:15:41.016554 396 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260619 21:15:41.024724 1613 catalog_manager.cc:1347] Generated new cluster ID: 0a4d3766bf294e5c9d1fcfdb50693f6b
I20260619 21:15:41.025036 1613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 21:15:41.049240 1613 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 21:15:41.051059 1613 catalog_manager.cc:1504] Loading token signing keys...
I20260619 21:15:41.063323 1613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Generated new TSK 0
I20260619 21:15:41.064069 1613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 21:15:41.083079 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:41.089413 1623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:41.090775 1624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:41.092656 1626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:41.092818 396 server_base.cc:1034] running on GCE node
I20260619 21:15:41.094063 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:41.094240 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:41.094363 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903741094354 us; error 0 us; skew 500 ppm
I20260619 21:15:41.094836 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:41.097311 396 webserver.cc:460] Webserver started at http://127.0.99.1:36507/ using document root <none> and password file <none>
I20260619 21:15:41.097749 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:41.097900 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:41.098109 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:41.099225 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-0-root/instance:
uuid: "884c214bb0a7455d91701ec388635e74"
format_stamp: "Formatted at 2026-06-19 21:15:41 on dist-test-slave-nlqf"
I20260619 21:15:41.104027 396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260619 21:15:41.107398 1631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.108314 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 21:15:41.108642 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-0-root
uuid: "884c214bb0a7455d91701ec388635e74"
format_stamp: "Formatted at 2026-06-19 21:15:41 on dist-test-slave-nlqf"
I20260619 21:15:41.109005 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:41.126590 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:41.127887 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:41.129405 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:15:41.131770 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:15:41.131939 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.132246 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:15:41.132467 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.178462 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.1:45637
I20260619 21:15:41.178592 1693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.1:45637 every 8 connection(s)
I20260619 21:15:41.183220 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:41.191417 1698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:41.191790 1699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:41.194782 1701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:41.196890 396 server_base.cc:1034] running on GCE node
I20260619 21:15:41.198729 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:41.198994 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:41.199208 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903741199189 us; error 0 us; skew 500 ppm
I20260619 21:15:41.199885 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:41.199952 1694 heartbeater.cc:347] Connected to a master server at 127.0.99.62:42975
I20260619 21:15:41.200410 1694 heartbeater.cc:464] Registering TS with master...
I20260619 21:15:41.201494 1694 heartbeater.cc:511] Master 127.0.99.62:42975 requested a full tablet report, sending...
I20260619 21:15:41.202960 396 webserver.cc:460] Webserver started at http://127.0.99.2:39255/ using document root <none> and password file <none>
I20260619 21:15:41.203649 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:41.203879 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:41.204063 1567 ts_manager.cc:194] Registered new tserver with Master: 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:15:41.204185 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:41.205610 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-1-root/instance:
uuid: "d53146739e3240239c0a3d5ea90e07fe"
format_stamp: "Formatted at 2026-06-19 21:15:41 on dist-test-slave-nlqf"
I20260619 21:15:41.206243 1567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37710
I20260619 21:15:41.211328 396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 21:15:41.214854 1706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.215552 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 21:15:41.215855 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-1-root
uuid: "d53146739e3240239c0a3d5ea90e07fe"
format_stamp: "Formatted at 2026-06-19 21:15:41 on dist-test-slave-nlqf"
I20260619 21:15:41.216130 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:41.238382 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:41.239665 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:41.241221 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:15:41.243557 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:15:41.243784 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.244028 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:15:41.244200 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.287190 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.2:42475
I20260619 21:15:41.287303 1768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.2:42475 every 8 connection(s)
I20260619 21:15:41.292181 396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 21:15:41.301177 1773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 21:15:41.303323 1775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:41.303617 396 server_base.cc:1034] running on GCE node
W20260619 21:15:41.303561 1772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 21:15:41.305065 396 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260619 21:15:41.305356 396 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260619 21:15:41.305558 396 hybrid_clock.cc:648] HybridClock initialized: now 1781903741305544 us; error 0 us; skew 500 ppm
I20260619 21:15:41.306296 396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 21:15:41.308324 1769 heartbeater.cc:347] Connected to a master server at 127.0.99.62:42975
I20260619 21:15:41.308771 1769 heartbeater.cc:464] Registering TS with master...
I20260619 21:15:41.309208 396 webserver.cc:460] Webserver started at http://127.0.99.3:42547/ using document root <none> and password file <none>
I20260619 21:15:41.309844 396 fs_manager.cc:359] Metadata directory not provided
I20260619 21:15:41.309758 1769 heartbeater.cc:511] Master 127.0.99.62:42975 requested a full tablet report, sending...
I20260619 21:15:41.310197 396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 21:15:41.310549 396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 21:15:41.312183 1567 ts_manager.cc:194] Registered new tserver with Master: d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:41.312187 396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-2-root/instance:
uuid: "b2e7f02778ec43bdbca03dd013687c76"
format_stamp: "Formatted at 2026-06-19 21:15:41 on dist-test-slave-nlqf"
I20260619 21:15:41.314608 1567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37726
I20260619 21:15:41.318394 396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 21:15:41.321787 1780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.322535 396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 21:15:41.322832 396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-2-root
uuid: "b2e7f02778ec43bdbca03dd013687c76"
format_stamp: "Formatted at 2026-06-19 21:15:41 on dist-test-slave-nlqf"
I20260619 21:15:41.323112 396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 21:15:41.355389 396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 21:15:41.356719 396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 21:15:41.358263 396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 21:15:41.360562 396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 21:15:41.360778 396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.361064 396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 21:15:41.361224 396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 21:15:41.405072 396 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.3:45591
I20260619 21:15:41.405135 1842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.3:45591 every 8 connection(s)
I20260619 21:15:41.421728 1843 heartbeater.cc:347] Connected to a master server at 127.0.99.62:42975
I20260619 21:15:41.422129 1843 heartbeater.cc:464] Registering TS with master...
I20260619 21:15:41.422880 1843 heartbeater.cc:511] Master 127.0.99.62:42975 requested a full tablet report, sending...
I20260619 21:15:41.425068 1567 ts_manager.cc:194] Registered new tserver with Master: b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:41.425275 396 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01696734s
I20260619 21:15:41.426868 1567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37734
I20260619 21:15:41.427637 396 test_util.cc:268] Using random seed: -880271458
I20260619 21:15:41.454226 1567 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37744:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 21:15:41.460631 1567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 21:15:41.677706 1734 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1e21185de74e3da01e18ff99572a08 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260619 21:15:41.679237 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1e21185de74e3da01e18ff99572a08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.679527 1733 tablet_service.cc:1467] Processing CreateTablet for tablet ef644e16cf5b46b68fde33b180432159 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260619 21:15:41.682158 1732 tablet_service.cc:1467] Processing CreateTablet for tablet 13c1f262670941baa9933b28f3c67fc2 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260619 21:15:41.683315 1731 tablet_service.cc:1467] Processing CreateTablet for tablet dfef072e2311405996f05452211cddca (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260619 21:15:41.702623 1651 tablet_service.cc:1467] Processing CreateTablet for tablet f756a9c8b10a4992950255915336bb2c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260619 21:15:41.704195 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f756a9c8b10a4992950255915336bb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.705101 1650 tablet_service.cc:1467] Processing CreateTablet for tablet efba5ed7ac544242953fd0c25465b088 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260619 21:15:41.706498 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efba5ed7ac544242953fd0c25465b088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.680909 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef644e16cf5b46b68fde33b180432159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.707955 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13c1f262670941baa9933b28f3c67fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.684422 1730 tablet_service.cc:1467] Processing CreateTablet for tablet 8e9a51e66e0646eeb7161c0cb670ae77 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260619 21:15:41.711934 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e9a51e66e0646eeb7161c0cb670ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.685587 1729 tablet_service.cc:1467] Processing CreateTablet for tablet 32ea84a8ad12422f8ec0ede64598fa44 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260619 21:15:41.714430 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32ea84a8ad12422f8ec0ede64598fa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.690428 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 32ea84a8ad12422f8ec0ede64598fa44 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260619 21:15:41.688544 1659 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1e21185de74e3da01e18ff99572a08 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260619 21:15:41.694456 1655 tablet_service.cc:1467] Processing CreateTablet for tablet 8e9a51e66e0646eeb7161c0cb670ae77 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260619 21:15:41.695562 1653 tablet_service.cc:1467] Processing CreateTablet for tablet 68eaefe3f12e418dbd2cdd20c2a3e228 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260619 21:15:41.692279 1657 tablet_service.cc:1467] Processing CreateTablet for tablet 13c1f262670941baa9933b28f3c67fc2 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260619 21:15:41.693470 1656 tablet_service.cc:1467] Processing CreateTablet for tablet dfef072e2311405996f05452211cddca (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260619 21:15:41.699143 1727 tablet_service.cc:1467] Processing CreateTablet for tablet f756a9c8b10a4992950255915336bb2c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260619 21:15:41.726225 1807 tablet_service.cc:1467] Processing CreateTablet for tablet ef644e16cf5b46b68fde33b180432159 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260619 21:15:41.699920 1725 tablet_service.cc:1467] Processing CreateTablet for tablet cd1e5b0f2736497bb5ea639a3071f8a2 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260619 21:15:41.727574 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef644e16cf5b46b68fde33b180432159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.700306 1726 tablet_service.cc:1467] Processing CreateTablet for tablet efba5ed7ac544242953fd0c25465b088 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260619 21:15:41.729948 1806 tablet_service.cc:1467] Processing CreateTablet for tablet 13c1f262670941baa9933b28f3c67fc2 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260619 21:15:41.731268 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13c1f262670941baa9933b28f3c67fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.734014 1805 tablet_service.cc:1467] Processing CreateTablet for tablet dfef072e2311405996f05452211cddca (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260619 21:15:41.735312 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfef072e2311405996f05452211cddca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.738162 1804 tablet_service.cc:1467] Processing CreateTablet for tablet 8e9a51e66e0646eeb7161c0cb670ae77 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260619 21:15:41.739464 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e9a51e66e0646eeb7161c0cb670ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.686734 1728 tablet_service.cc:1467] Processing CreateTablet for tablet 68eaefe3f12e418dbd2cdd20c2a3e228 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260619 21:15:41.691160 1658 tablet_service.cc:1467] Processing CreateTablet for tablet ef644e16cf5b46b68fde33b180432159 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260619 21:15:41.696738 1652 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb9bc6419d14ea8b9834eea6fafc54a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260619 21:15:41.697190 1858 tablet_bootstrap.cc:492] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:41.718957 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32ea84a8ad12422f8ec0ede64598fa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.725077 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1e21185de74e3da01e18ff99572a08 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260619 21:15:41.754359 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1e21185de74e3da01e18ff99572a08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.758808 1730 tablet_service.cc:1467] Processing CreateTablet for tablet 84c655fb144a4c11b8902e23b4fac063 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260619 21:15:41.718086 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfef072e2311405996f05452211cddca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.761965 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f756a9c8b10a4992950255915336bb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.762583 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13c1f262670941baa9933b28f3c67fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.763234 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef644e16cf5b46b68fde33b180432159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.758749 1732 tablet_service.cc:1467] Processing CreateTablet for tablet c28152c520fe41e5be2905e6e3b67b8a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260619 21:15:41.763845 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68eaefe3f12e418dbd2cdd20c2a3e228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.763721 1729 tablet_service.cc:1467] Processing CreateTablet for tablet 4d7929378ea14d5fbd2b7c633537d332 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260619 21:15:41.764588 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c28152c520fe41e5be2905e6e3b67b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.765170 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1e21185de74e3da01e18ff99572a08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.767329 1733 tablet_service.cc:1467] Processing CreateTablet for tablet c8a530b7f9264d519f65caf431a0a478 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260619 21:15:41.767272 1801 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb9bc6419d14ea8b9834eea6fafc54a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260619 21:15:41.768839 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb9bc6419d14ea8b9834eea6fafc54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.770169 1799 tablet_service.cc:1467] Processing CreateTablet for tablet efba5ed7ac544242953fd0c25465b088 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260619 21:15:41.771589 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efba5ed7ac544242953fd0c25465b088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.771332 1802 tablet_service.cc:1467] Processing CreateTablet for tablet 68eaefe3f12e418dbd2cdd20c2a3e228 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260619 21:15:41.772641 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68eaefe3f12e418dbd2cdd20c2a3e228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.772758 1800 tablet_service.cc:1467] Processing CreateTablet for tablet f756a9c8b10a4992950255915336bb2c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260619 21:15:41.774145 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f756a9c8b10a4992950255915336bb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.770169 1803 tablet_service.cc:1467] Processing CreateTablet for tablet 32ea84a8ad12422f8ec0ede64598fa44 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260619 21:15:41.777060 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32ea84a8ad12422f8ec0ede64598fa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.777330 1726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efba5ed7ac544242953fd0c25465b088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.778091 1651 tablet_service.cc:1467] Processing CreateTablet for tablet cd1e5b0f2736497bb5ea639a3071f8a2 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260619 21:15:41.779352 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1e5b0f2736497bb5ea639a3071f8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.781351 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb9bc6419d14ea8b9834eea6fafc54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.783619 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68eaefe3f12e418dbd2cdd20c2a3e228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.785131 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a530b7f9264d519f65caf431a0a478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.792845 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e9a51e66e0646eeb7161c0cb670ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.797356 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1e5b0f2736497bb5ea639a3071f8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.801151 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfef072e2311405996f05452211cddca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.801275 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c655fb144a4c11b8902e23b4fac063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.698488 1734 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb9bc6419d14ea8b9834eea6fafc54a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260619 21:15:41.818406 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb9bc6419d14ea8b9834eea6fafc54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.837096 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d7929378ea14d5fbd2b7c633537d332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.850831 1858 tablet_bootstrap.cc:654] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:41.858112 1650 tablet_service.cc:1467] Processing CreateTablet for tablet c8a530b7f9264d519f65caf431a0a478 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260619 21:15:41.859712 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a530b7f9264d519f65caf431a0a478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.866300 1859 tablet_bootstrap.cc:492] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:41.886833 1727 tablet_service.cc:1467] Processing CreateTablet for tablet f9c4f216cdd7450b9659feeea5fd2bf9 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260619 21:15:41.888260 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c4f216cdd7450b9659feeea5fd2bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.897564 1808 tablet_service.cc:1467] Processing CreateTablet for tablet cd1e5b0f2736497bb5ea639a3071f8a2 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260619 21:15:41.897967 1803 tablet_service.cc:1467] Processing CreateTablet for tablet c8a530b7f9264d519f65caf431a0a478 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260619 21:15:41.898932 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1e5b0f2736497bb5ea639a3071f8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.900740 1860 tablet_bootstrap.cc:492] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:41.904043 1730 tablet_service.cc:1467] Processing CreateTablet for tablet 78cb98bb4e2642acb6448e5f5ccdb1ff (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260619 21:15:41.909911 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cb98bb4e2642acb6448e5f5ccdb1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.910035 1799 tablet_service.cc:1467] Processing CreateTablet for tablet c28152c520fe41e5be2905e6e3b67b8a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260619 21:15:41.911996 1805 tablet_service.cc:1467] Processing CreateTablet for tablet 84c655fb144a4c11b8902e23b4fac063 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260619 21:15:41.912978 1801 tablet_service.cc:1467] Processing CreateTablet for tablet 4d7929378ea14d5fbd2b7c633537d332 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260619 21:15:41.916831 1732 tablet_service.cc:1467] Processing CreateTablet for tablet 34c673e979964335b28709efdae7ece1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260619 21:15:41.889710 1859 tablet_bootstrap.cc:654] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:41.917181 1802 tablet_service.cc:1467] Processing CreateTablet for tablet 044f6297187147aba0e5ec306cc3840a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260619 21:15:41.900310 1658 tablet_service.cc:1467] Processing CreateTablet for tablet c28152c520fe41e5be2905e6e3b67b8a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260619 21:15:41.923913 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a530b7f9264d519f65caf431a0a478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.923950 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c28152c520fe41e5be2905e6e3b67b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.924871 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c28152c520fe41e5be2905e6e3b67b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.926803 1650 tablet_service.cc:1467] Processing CreateTablet for tablet 4d7929378ea14d5fbd2b7c633537d332 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260619 21:15:41.930596 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d7929378ea14d5fbd2b7c633537d332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.932756 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c655fb144a4c11b8902e23b4fac063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.936003 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d7929378ea14d5fbd2b7c633537d332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.936908 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044f6297187147aba0e5ec306cc3840a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.939520 1800 tablet_service.cc:1467] Processing CreateTablet for tablet f9c4f216cdd7450b9659feeea5fd2bf9 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260619 21:15:41.939594 1651 tablet_service.cc:1467] Processing CreateTablet for tablet f9c4f216cdd7450b9659feeea5fd2bf9 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260619 21:15:41.917104 1659 tablet_service.cc:1467] Processing CreateTablet for tablet 84c655fb144a4c11b8902e23b4fac063 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260619 21:15:41.961177 1656 tablet_service.cc:1467] Processing CreateTablet for tablet 34c673e979964335b28709efdae7ece1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260619 21:15:41.967553 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c673e979964335b28709efdae7ece1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.977061 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c4f216cdd7450b9659feeea5fd2bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.983525 1733 tablet_service.cc:1467] Processing CreateTablet for tablet 24c647ebb0b64c158458c654b2462eeb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260619 21:15:41.985074 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24c647ebb0b64c158458c654b2462eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:41.943809 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 78cb98bb4e2642acb6448e5f5ccdb1ff (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260619 21:15:41.970358 1653 tablet_service.cc:1467] Processing CreateTablet for tablet 8133a49f53cd4e0c9bcb3b4c9d5da36d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260619 21:15:42.017923 1734 tablet_service.cc:1467] Processing CreateTablet for tablet 8133a49f53cd4e0c9bcb3b4c9d5da36d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260619 21:15:42.020221 1807 tablet_service.cc:1467] Processing CreateTablet for tablet 8133a49f53cd4e0c9bcb3b4c9d5da36d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260619 21:15:42.021073 1806 tablet_service.cc:1467] Processing CreateTablet for tablet 78cb98bb4e2642acb6448e5f5ccdb1ff (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260619 21:15:42.032361 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8133a49f53cd4e0c9bcb3b4c9d5da36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.033344 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cb98bb4e2642acb6448e5f5ccdb1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.023080 1804 tablet_service.cc:1467] Processing CreateTablet for tablet 34c673e979964335b28709efdae7ece1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260619 21:15:42.037724 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8133a49f53cd4e0c9bcb3b4c9d5da36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.038010 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c673e979964335b28709efdae7ece1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.039177 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c4f216cdd7450b9659feeea5fd2bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.028108 1652 tablet_service.cc:1467] Processing CreateTablet for tablet 24c647ebb0b64c158458c654b2462eeb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260619 21:15:42.047580 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c655fb144a4c11b8902e23b4fac063. 1 dirs total, 0 dirs full, 0 dirs failed
W20260619 21:15:42.049598 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:42.054068 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8133a49f53cd4e0c9bcb3b4c9d5da36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.058363 1727 tablet_service.cc:1467] Processing CreateTablet for tablet a192c7be8b2c49bda61f7d5588b3d246 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260619 21:15:42.059831 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a192c7be8b2c49bda61f7d5588b3d246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.060820 1858 tablet_bootstrap.cc:492] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:42.061357 1858 ts_tablet_manager.cc:1397] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.365s user 0.036s sys 0.028s
W20260619 21:15:42.061408 1554 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 61000.2us
I20260619 21:15:42.065297 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cb98bb4e2642acb6448e5f5ccdb1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.058445 1728 tablet_service.cc:1467] Processing CreateTablet for tablet 25c06c6619d04c58a5ed3f65b1c82f70 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260619 21:15:42.069808 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c06c6619d04c58a5ed3f65b1c82f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.069607 1657 tablet_service.cc:1467] Processing CreateTablet for tablet 044f6297187147aba0e5ec306cc3840a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260619 21:15:42.080673 1727 tablet_service.cc:1467] Processing CreateTablet for tablet da1270002c3c45988c3ac4fbeafaf019 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260619 21:15:42.081821 1859 tablet_bootstrap.cc:492] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:42.082052 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1270002c3c45988c3ac4fbeafaf019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.082361 1859 ts_tablet_manager.cc:1397] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.216s user 0.034s sys 0.017s
I20260619 21:15:42.083572 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c673e979964335b28709efdae7ece1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.085412 1859 raft_consensus.cc:348] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.086189 1859 raft_consensus.cc:374] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.086460 1859 raft_consensus.cc:729] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.087163 1859 consensus_queue.cc:260] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.098097 1730 tablet_service.cc:1467] Processing CreateTablet for tablet f0b6191588f14043884eceae0b64c41c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260619 21:15:42.099047 1727 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce345f630e142ac9300688d473743af (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260619 21:15:42.099567 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b6191588f14043884eceae0b64c41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.100330 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce345f630e142ac9300688d473743af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.064383 1858 raft_consensus.cc:348] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.100854 1858 raft_consensus.cc:374] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.101187 1858 raft_consensus.cc:729] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.101987 1858 consensus_queue.cc:260] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.110298 1859 ts_tablet_manager.cc:1428] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.028s user 0.008s sys 0.000s
W20260619 21:15:42.113101 1552 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 50972.6us
I20260619 21:15:42.117425 1860 tablet_bootstrap.cc:700] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Time spent opening tablet: real 0.105s user 0.003s sys 0.000s
I20260619 21:15:42.118079 1860 tablet_bootstrap.cc:654] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.123734 1729 tablet_service.cc:1467] Processing CreateTablet for tablet a3c98fc9dacf48e8994724fe2e303166 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260619 21:15:42.127775 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044f6297187147aba0e5ec306cc3840a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.129730 1694 heartbeater.cc:503] Master 127.0.99.62:42975 was elected leader, sending a full tablet report...
I20260619 21:15:42.132915 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24c647ebb0b64c158458c654b2462eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.142468 1803 tablet_service.cc:1467] Processing CreateTablet for tablet 24c647ebb0b64c158458c654b2462eeb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260619 21:15:42.143947 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24c647ebb0b64c158458c654b2462eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.144757 1859 tablet_bootstrap.cc:492] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:42.153149 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c98fc9dacf48e8994724fe2e303166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.154191 1731 tablet_service.cc:1467] Processing CreateTablet for tablet d3404667de13461aa4b3e79e8d6a64ee (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260619 21:15:42.156103 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3404667de13461aa4b3e79e8d6a64ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.128376 1726 tablet_service.cc:1467] Processing CreateTablet for tablet 044f6297187147aba0e5ec306cc3840a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260619 21:15:42.178018 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 25c06c6619d04c58a5ed3f65b1c82f70 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260619 21:15:42.179479 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c06c6619d04c58a5ed3f65b1c82f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.183073 1802 tablet_service.cc:1467] Processing CreateTablet for tablet a192c7be8b2c49bda61f7d5588b3d246 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
W20260619 21:15:42.183697 1695 tablet.cc:2369] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 21:15:42.186468 1653 tablet_service.cc:1467] Processing CreateTablet for tablet 25c06c6619d04c58a5ed3f65b1c82f70 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260619 21:15:42.186877 1655 tablet_service.cc:1467] Processing CreateTablet for tablet a192c7be8b2c49bda61f7d5588b3d246 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260619 21:15:42.186703 1805 tablet_service.cc:1467] Processing CreateTablet for tablet d3404667de13461aa4b3e79e8d6a64ee (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260619 21:15:42.187884 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c06c6619d04c58a5ed3f65b1c82f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.189496 1804 tablet_service.cc:1467] Processing CreateTablet for tablet da1270002c3c45988c3ac4fbeafaf019 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260619 21:15:42.194036 1732 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd2b54df9ba4dac85c2eb1387dd6f03 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260619 21:15:42.211047 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3404667de13461aa4b3e79e8d6a64ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.215425 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1270002c3c45988c3ac4fbeafaf019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.211264 1859 tablet_bootstrap.cc:654] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.200713 1656 tablet_service.cc:1467] Processing CreateTablet for tablet d3404667de13461aa4b3e79e8d6a64ee (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260619 21:15:42.222280 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a192c7be8b2c49bda61f7d5588b3d246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.241461 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3404667de13461aa4b3e79e8d6a64ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.236716 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a192c7be8b2c49bda61f7d5588b3d246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.240396 1726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044f6297187147aba0e5ec306cc3840a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.246258 1863 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:42.249502 1769 heartbeater.cc:503] Master 127.0.99.62:42975 was elected leader, sending a full tablet report...
I20260619 21:15:42.250834 1652 tablet_service.cc:1467] Processing CreateTablet for tablet da1270002c3c45988c3ac4fbeafaf019 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260619 21:15:42.249411 1863 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.216204 1725 tablet_service.cc:1467] Processing CreateTablet for tablet 516fa114155140f6b508f311c4971c08 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260619 21:15:42.260128 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fd2b54df9ba4dac85c2eb1387dd6f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.260844 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 516fa114155140f6b508f311c4971c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.274708 1807 tablet_service.cc:1467] Processing CreateTablet for tablet f0b6191588f14043884eceae0b64c41c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260619 21:15:42.275987 1858 ts_tablet_manager.cc:1428] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.214s user 0.026s sys 0.046s
I20260619 21:15:42.276166 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b6191588f14043884eceae0b64c41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.281714 1863 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:42.292109 1658 tablet_service.cc:1467] Processing CreateTablet for tablet f0b6191588f14043884eceae0b64c41c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260619 21:15:42.292876 1729 tablet_service.cc:1467] Processing CreateTablet for tablet deaff01bc5d54f808990dbcbaf46c0b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260619 21:15:42.294943 1731 tablet_service.cc:1467] Processing CreateTablet for tablet b1857ce01be64d68891414a298ac40af (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260619 21:15:42.296144 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deaff01bc5d54f808990dbcbaf46c0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.300108 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1857ce01be64d68891414a298ac40af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.302639 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1270002c3c45988c3ac4fbeafaf019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.302579 1858 tablet_bootstrap.cc:492] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:42.327879 1799 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce345f630e142ac9300688d473743af (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260619 21:15:42.331329 1864 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:42.333250 1860 tablet_bootstrap.cc:492] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:42.332912 1864 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.333742 1860 ts_tablet_manager.cc:1397] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.433s user 0.021s sys 0.013s
I20260619 21:15:42.339167 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b6191588f14043884eceae0b64c41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.342782 1864 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:42.383414 1651 tablet_service.cc:1467] Processing CreateTablet for tablet a3c98fc9dacf48e8994724fe2e303166 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260619 21:15:42.383862 1800 tablet_service.cc:1467] Processing CreateTablet for tablet a3c98fc9dacf48e8994724fe2e303166 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260619 21:15:42.357611 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce345f630e142ac9300688d473743af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.389729 1860 raft_consensus.cc:348] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.390872 1860 raft_consensus.cc:374] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.391319 1860 raft_consensus.cc:729] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.404749 1858 tablet_bootstrap.cc:654] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.411387 1657 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd2b54df9ba4dac85c2eb1387dd6f03 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260619 21:15:42.412096 1727 tablet_service.cc:1467] Processing CreateTablet for tablet 6a49a9579b0a40fdbfe77c1e282f1800 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260619 21:15:42.392244 1860 consensus_queue.cc:260] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.413415 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a49a9579b0a40fdbfe77c1e282f1800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.388404 1725 tablet_service.cc:1467] Processing CreateTablet for tablet 9980ccca53c3424db1585f12669d6d58 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260619 21:15:42.384853 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c98fc9dacf48e8994724fe2e303166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.430537 1843 heartbeater.cc:503] Master 127.0.99.62:42975 was elected leader, sending a full tablet report...
I20260619 21:15:42.343570 1659 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce345f630e142ac9300688d473743af (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260619 21:15:42.434152 1650 tablet_service.cc:1467] Processing CreateTablet for tablet 516fa114155140f6b508f311c4971c08 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260619 21:15:42.435837 1801 tablet_service.cc:1467] Processing CreateTablet for tablet 516fa114155140f6b508f311c4971c08 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260619 21:15:42.437399 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9980ccca53c3424db1585f12669d6d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.444657 1654 tablet_service.cc:1467] Processing CreateTablet for tablet deaff01bc5d54f808990dbcbaf46c0b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260619 21:15:42.451385 1653 tablet_service.cc:1467] Processing CreateTablet for tablet b1857ce01be64d68891414a298ac40af (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260619 21:15:42.452566 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c98fc9dacf48e8994724fe2e303166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.453491 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd2b54df9ba4dac85c2eb1387dd6f03 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260619 21:15:42.454622 1658 tablet_service.cc:1467] Processing CreateTablet for tablet 9980ccca53c3424db1585f12669d6d58 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260619 21:15:42.456907 1806 tablet_service.cc:1467] Processing CreateTablet for tablet deaff01bc5d54f808990dbcbaf46c0b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260619 21:15:42.468058 1859 tablet_bootstrap.cc:492] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:42.476547 1859 ts_tablet_manager.cc:1397] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.332s user 0.015s sys 0.021s
I20260619 21:15:42.472150 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 516fa114155140f6b508f311c4971c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.412096 1733 tablet_service.cc:1467] Processing CreateTablet for tablet bfe558fbf8e24ddfbe844d69487cce1d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260619 21:15:42.473690 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9980ccca53c3424db1585f12669d6d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.481822 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe558fbf8e24ddfbe844d69487cce1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.482369 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deaff01bc5d54f808990dbcbaf46c0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.484032 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fd2b54df9ba4dac85c2eb1387dd6f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.485221 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fd2b54df9ba4dac85c2eb1387dd6f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.488749 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce345f630e142ac9300688d473743af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.492739 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deaff01bc5d54f808990dbcbaf46c0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.497638 1859 raft_consensus.cc:348] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.500768 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1857ce01be64d68891414a298ac40af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.512125 1805 tablet_service.cc:1467] Processing CreateTablet for tablet 9980ccca53c3424db1585f12669d6d58 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260619 21:15:42.505311 1859 raft_consensus.cc:374] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.516824 1859 raft_consensus.cc:729] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.517742 1859 consensus_queue.cc:260] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.518735 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:42.523430 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 516fa114155140f6b508f311c4971c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.530028 1803 tablet_service.cc:1467] Processing CreateTablet for tablet b1857ce01be64d68891414a298ac40af (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260619 21:15:42.535462 1859 ts_tablet_manager.cc:1428] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.055s user 0.007s sys 0.000s
I20260619 21:15:42.536512 1859 tablet_bootstrap.cc:492] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:42.538559 1730 tablet_service.cc:1467] Processing CreateTablet for tablet ccfc3736d63b4f80ab5f68b26ab104a0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260619 21:15:42.545967 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:42.557039 1658 tablet_service.cc:1467] Processing CreateTablet for tablet 6a49a9579b0a40fdbfe77c1e282f1800 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260619 21:15:42.557710 1635 leader_election.cc:341] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:42.558679 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a49a9579b0a40fdbfe77c1e282f1800. 1 dirs total, 0 dirs full, 0 dirs failed
W20260619 21:15:42.546039 1708 leader_election.cc:341] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:42.570361 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccfc3736d63b4f80ab5f68b26ab104a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.570175 1732 tablet_service.cc:1467] Processing CreateTablet for tablet 6333f19ae36b48459bbfa0490f45d128 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260619 21:15:42.570963 1802 tablet_service.cc:1467] Processing CreateTablet for tablet bfe558fbf8e24ddfbe844d69487cce1d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260619 21:15:42.588487 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9980ccca53c3424db1585f12669d6d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.588606 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:42.589195 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe558fbf8e24ddfbe844d69487cce1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.571003 1728 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed3117ca10a4413872e39fb4535aec7 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260619 21:15:42.590237 1710 leader_election.cc:341] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:42.590713 1710 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:42.591603 1859 tablet_bootstrap.cc:654] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.591853 1864 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:42.576172 1734 tablet_service.cc:1467] Processing CreateTablet for tablet 6984e19be5654fef830977bff30f643e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260619 21:15:42.588096 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6333f19ae36b48459bbfa0490f45d128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.593096 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6984e19be5654fef830977bff30f643e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.593643 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed3117ca10a4413872e39fb4535aec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.599977 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:42.601735 1634 leader_election.cc:341] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:42.602200 1634 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:42.604389 1863 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:42.605010 1725 tablet_service.cc:1467] Processing CreateTablet for tablet 0d45d883f7fa4dd285976666f28c0770 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260619 21:15:42.608541 1858 tablet_bootstrap.cc:492] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:42.613518 1858 ts_tablet_manager.cc:1397] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.311s user 0.019s sys 0.011s
I20260619 21:15:42.607792 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d45d883f7fa4dd285976666f28c0770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.609381 1726 tablet_service.cc:1467] Processing CreateTablet for tablet ffcc16880ccb4a85a786728439b64cc0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260619 21:15:42.622416 1726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcc16880ccb4a85a786728439b64cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.624455 1860 ts_tablet_manager.cc:1428] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.290s user 0.017s sys 0.002s
I20260619 21:15:42.610977 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1857ce01be64d68891414a298ac40af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.625504 1860 tablet_bootstrap.cc:492] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:42.621176 1656 tablet_service.cc:1467] Processing CreateTablet for tablet bfe558fbf8e24ddfbe844d69487cce1d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260619 21:15:42.626547 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe558fbf8e24ddfbe844d69487cce1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.623795 1799 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed3117ca10a4413872e39fb4535aec7 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260619 21:15:42.629093 1804 tablet_service.cc:1467] Processing CreateTablet for tablet 6a49a9579b0a40fdbfe77c1e282f1800 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260619 21:15:42.638633 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 6333f19ae36b48459bbfa0490f45d128 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260619 21:15:42.616390 1858 raft_consensus.cc:348] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.639206 1657 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed3117ca10a4413872e39fb4535aec7 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260619 21:15:42.639705 1858 raft_consensus.cc:374] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.635418 1733 tablet_service.cc:1467] Processing CreateTablet for tablet 225b6f9e181a43de8dceeb75cbb1c01d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260619 21:15:42.642490 1729 tablet_service.cc:1467] Processing CreateTablet for tablet 2aedd754fcc34eecbb29d5cccbb9153a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260619 21:15:42.643033 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225b6f9e181a43de8dceeb75cbb1c01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.654537 1727 tablet_service.cc:1467] Processing CreateTablet for tablet 1764e19eb06140f8bbdb1c3aab252d9a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260619 21:15:42.642999 1731 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe8c95af50740a783386324c1649be0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260619 21:15:42.655894 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1764e19eb06140f8bbdb1c3aab252d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.660658 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe8c95af50740a783386324c1649be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.664638 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a49a9579b0a40fdbfe77c1e282f1800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.665670 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aedd754fcc34eecbb29d5cccbb9153a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.669741 1654 tablet_service.cc:1467] Processing CreateTablet for tablet ccfc3736d63b4f80ab5f68b26ab104a0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260619 21:15:42.651585 1728 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae5f9625019495d85bc13c82d7f621b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260619 21:15:42.674300 1652 tablet_service.cc:1467] Processing CreateTablet for tablet 6333f19ae36b48459bbfa0490f45d128 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260619 21:15:42.674933 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae5f9625019495d85bc13c82d7f621b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.640054 1858 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.676231 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed3117ca10a4413872e39fb4535aec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.680573 1801 tablet_service.cc:1467] Processing CreateTablet for tablet ccfc3736d63b4f80ab5f68b26ab104a0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260619 21:15:42.681564 1859 tablet_bootstrap.cc:492] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:42.682075 1859 ts_tablet_manager.cc:1397] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.146s user 0.014s sys 0.004s
I20260619 21:15:42.683917 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6333f19ae36b48459bbfa0490f45d128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.689286 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccfc3736d63b4f80ab5f68b26ab104a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.693689 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccfc3736d63b4f80ab5f68b26ab104a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.694810 1733 tablet_service.cc:1467] Processing CreateTablet for tablet 84119be03678450983c7d5a686bd3c86 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260619 21:15:42.696103 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84119be03678450983c7d5a686bd3c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.697506 1858 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.690312 1860 tablet_bootstrap.cc:654] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.689353 1859 raft_consensus.cc:348] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.717512 1859 raft_consensus.cc:374] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.717905 1859 raft_consensus.cc:729] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.718431 1858 ts_tablet_manager.cc:1428] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.105s user 0.006s sys 0.002s
I20260619 21:15:42.718777 1859 consensus_queue.cc:260] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.719455 1858 tablet_bootstrap.cc:492] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:42.721133 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6333f19ae36b48459bbfa0490f45d128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.698508 1806 tablet_service.cc:1467] Processing CreateTablet for tablet ffcc16880ccb4a85a786728439b64cc0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260619 21:15:42.726477 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcc16880ccb4a85a786728439b64cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.727372 1807 tablet_service.cc:1467] Processing CreateTablet for tablet 6984e19be5654fef830977bff30f643e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260619 21:15:42.729305 1655 tablet_service.cc:1467] Processing CreateTablet for tablet ffcc16880ccb4a85a786728439b64cc0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260619 21:15:42.729887 1875 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:42.730623 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcc16880ccb4a85a786728439b64cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.730400 1875 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.730734 1659 tablet_service.cc:1467] Processing CreateTablet for tablet 6984e19be5654fef830977bff30f643e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260619 21:15:42.730926 1858 tablet_bootstrap.cc:654] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.733182 1859 ts_tablet_manager.cc:1428] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.051s user 0.008s sys 0.002s
I20260619 21:15:42.712852 1803 tablet_service.cc:1467] Processing CreateTablet for tablet 0d45d883f7fa4dd285976666f28c0770 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260619 21:15:42.733784 1653 tablet_service.cc:1467] Processing CreateTablet for tablet 0d45d883f7fa4dd285976666f28c0770 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260619 21:15:42.734848 1875 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:42.736701 1805 tablet_service.cc:1467] Processing CreateTablet for tablet 225b6f9e181a43de8dceeb75cbb1c01d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260619 21:15:42.747886 1731 tablet_service.cc:1467] Processing CreateTablet for tablet 5e50dfc7ff65482ab6eb4b18b5ae06bb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260619 21:15:42.744416 1734 tablet_service.cc:1467] Processing CreateTablet for tablet fc672e451a9f4f6eb43430477e3acd1b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260619 21:15:42.757848 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc672e451a9f4f6eb43430477e3acd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.758105 1800 tablet_service.cc:1467] Processing CreateTablet for tablet 2aedd754fcc34eecbb29d5cccbb9153a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260619 21:15:42.769632 1859 tablet_bootstrap.cc:492] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:42.773869 1802 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe8c95af50740a783386324c1649be0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260619 21:15:42.783433 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6984e19be5654fef830977bff30f643e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.803625 1650 tablet_service.cc:1467] Processing CreateTablet for tablet 225b6f9e181a43de8dceeb75cbb1c01d (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260619 21:15:42.806746 1733 tablet_service.cc:1467] Processing CreateTablet for tablet e3b2adcb38244b60bdd7ae5d5642f641 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260619 21:15:42.809001 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b2adcb38244b60bdd7ae5d5642f641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.824057 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe8c95af50740a783386324c1649be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.830921 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aedd754fcc34eecbb29d5cccbb9153a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.834658 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225b6f9e181a43de8dceeb75cbb1c01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.806797 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6984e19be5654fef830977bff30f643e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.840320 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d45d883f7fa4dd285976666f28c0770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.841377 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e50dfc7ff65482ab6eb4b18b5ae06bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.841996 1728 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae81d060b714ae98341d6b5e2669f3f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260619 21:15:42.843309 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae81d060b714ae98341d6b5e2669f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.840179 1801 tablet_service.cc:1467] Processing CreateTablet for tablet 1764e19eb06140f8bbdb1c3aab252d9a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260619 21:15:42.857272 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d45d883f7fa4dd285976666f28c0770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.859763 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225b6f9e181a43de8dceeb75cbb1c01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.876579 1651 tablet_service.cc:1467] Processing CreateTablet for tablet 1764e19eb06140f8bbdb1c3aab252d9a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260619 21:15:42.877228 1658 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe8c95af50740a783386324c1649be0 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260619 21:15:42.878077 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1764e19eb06140f8bbdb1c3aab252d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.824928 1730 tablet_service.cc:1467] Processing CreateTablet for tablet 0dea6228e49c44b9889aa709158ef557 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260619 21:15:42.886128 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dea6228e49c44b9889aa709158ef557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.869132 1727 tablet_service.cc:1467] Processing CreateTablet for tablet cfa69d158f2b471e8638824d3d9c757b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260619 21:15:42.897796 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa69d158f2b471e8638824d3d9c757b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.898419 1657 tablet_service.cc:1467] Processing CreateTablet for tablet fc672e451a9f4f6eb43430477e3acd1b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260619 21:15:42.878377 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed3117ca10a4413872e39fb4535aec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.880295 1656 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae5f9625019495d85bc13c82d7f621b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260619 21:15:42.884171 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 84119be03678450983c7d5a686bd3c86 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260619 21:15:42.866901 1725 tablet_service.cc:1467] Processing CreateTablet for tablet 246c3d015ec242eca8bfaf5a076692b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260619 21:15:42.900271 1732 tablet_service.cc:1467] Processing CreateTablet for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260619 21:15:42.901258 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246c3d015ec242eca8bfaf5a076692b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.812898 1652 tablet_service.cc:1467] Processing CreateTablet for tablet 2aedd754fcc34eecbb29d5cccbb9153a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260619 21:15:42.905218 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.866943 1858 tablet_bootstrap.cc:492] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:42.906142 1858 ts_tablet_manager.cc:1397] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.187s user 0.008s sys 0.008s
I20260619 21:15:42.909238 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe8c95af50740a783386324c1649be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.869158 1860 tablet_bootstrap.cc:492] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:42.910244 1860 ts_tablet_manager.cc:1397] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.285s user 0.035s sys 0.045s
I20260619 21:15:42.913710 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1764e19eb06140f8bbdb1c3aab252d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.914394 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aedd754fcc34eecbb29d5cccbb9153a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.915218 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae5f9625019495d85bc13c82d7f621b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.915987 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84119be03678450983c7d5a686bd3c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.918427 1726 tablet_service.cc:1467] Processing CreateTablet for tablet b249d6b40f6943e3bb7235b7df99e1b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260619 21:15:42.926630 1860 raft_consensus.cc:348] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.927505 1860 raft_consensus.cc:374] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.927793 1860 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.928391 1726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b249d6b40f6943e3bb7235b7df99e1b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.939800 1729 tablet_service.cc:1467] Processing CreateTablet for tablet d80394dabe604f8898e7b24149fc624f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260619 21:15:42.928876 1858 raft_consensus.cc:348] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.941123 1858 raft_consensus.cc:374] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:42.941237 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80394dabe604f8898e7b24149fc624f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.943789 1655 tablet_service.cc:1467] Processing CreateTablet for tablet 5e50dfc7ff65482ab6eb4b18b5ae06bb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260619 21:15:42.933949 1804 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae5f9625019495d85bc13c82d7f621b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260619 21:15:42.935672 1859 tablet_bootstrap.cc:654] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:42.947716 1863 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:42.948179 1863 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.949465 1858 raft_consensus.cc:729] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:42.950682 1863 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:42.950822 1858 consensus_queue.cc:260] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:42.952375 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:42.954000 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:42.954964 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc672e451a9f4f6eb43430477e3acd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.957432 1744 raft_consensus.cc:2454] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:42.960446 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae5f9625019495d85bc13c82d7f621b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:42.961844 1634 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:42.962721 1863 raft_consensus.cc:2781] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:42.963055 1863 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:42.963359 1863 raft_consensus.cc:3037] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:42.965526 1635 leader_election.cc:341] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:42.970954 1860 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.013895 1875 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.014490 1875 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.017673 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e50dfc7ff65482ab6eb4b18b5ae06bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.019134 1875 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:42.999931 1860 ts_tablet_manager.cc:1428] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.089s user 0.005s sys 0.003s
I20260619 21:15:43.036981 1860 tablet_bootstrap.cc:492] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.042249 1806 tablet_service.cc:1467] Processing CreateTablet for tablet 84119be03678450983c7d5a686bd3c86 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260619 21:15:43.043756 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84119be03678450983c7d5a686bd3c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.046452 1860 tablet_bootstrap.cc:654] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:43.052151 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:43.090798 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.052559 1650 tablet_service.cc:1467] Processing CreateTablet for tablet e3b2adcb38244b60bdd7ae5d5642f641 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260619 21:15:43.093575 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:43.093796 1783 leader_election.cc:341] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.057739 1808 tablet_service.cc:1467] Processing CreateTablet for tablet fc672e451a9f4f6eb43430477e3acd1b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260619 21:15:43.095633 1727 tablet_service.cc:1467] Processing CreateTablet for tablet bb9b500bf9a24148b2fbcd762b549521 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260619 21:15:43.096167 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.096865 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb9b500bf9a24148b2fbcd762b549521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.097683 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:43.099056 1783 leader_election.cc:341] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.096174 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc672e451a9f4f6eb43430477e3acd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.099267 1659 tablet_service.cc:1467] Processing CreateTablet for tablet 0dea6228e49c44b9889aa709158ef557 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260619 21:15:43.101934 1656 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae81d060b714ae98341d6b5e2669f3f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260619 21:15:43.103135 1802 tablet_service.cc:1467] Processing CreateTablet for tablet 5e50dfc7ff65482ab6eb4b18b5ae06bb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260619 21:15:43.071738 1858 ts_tablet_manager.cc:1428] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.165s user 0.009s sys 0.001s
W20260619 21:15:43.113036 1782 leader_election.cc:341] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.120086 1883 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.119938 1782 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:43.120543 1883 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:43.121531 1782 leader_election.cc:341] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.121953 1782 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:43.122702 1883 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.122892 1875 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.122773 1734 tablet_service.cc:1467] Processing CreateTablet for tablet 692d38d1f19c468eae7077dde218a62f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260619 21:15:43.123572 1875 raft_consensus.cc:2726] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.123907 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b2adcb38244b60bdd7ae5d5642f641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.124361 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692d38d1f19c468eae7077dde218a62f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.125936 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.127128 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.127815 1803 tablet_service.cc:1467] Processing CreateTablet for tablet e3b2adcb38244b60bdd7ae5d5642f641 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260619 21:15:43.128880 1634 leader_election.cc:341] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.131016 1807 tablet_service.cc:1467] Processing CreateTablet for tablet 0dea6228e49c44b9889aa709158ef557 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260619 21:15:43.132328 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dea6228e49c44b9889aa709158ef557. 1 dirs total, 0 dirs full, 0 dirs failed
W20260619 21:15:43.133415 1635 leader_election.cc:341] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.134080 1635 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:43.134554 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae81d060b714ae98341d6b5e2669f3f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260619 21:15:43.135811 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae81d060b714ae98341d6b5e2669f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.137670 1859 tablet_bootstrap.cc:492] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.138154 1859 ts_tablet_manager.cc:1397] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.369s user 0.020s sys 0.020s
I20260619 21:15:43.140450 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dea6228e49c44b9889aa709158ef557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.143028 1883 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.149109 1864 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.150499 1864 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.152606 1864 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.153525 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae81d060b714ae98341d6b5e2669f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.154464 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:43.156318 1710 leader_election.cc:341] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.156246 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:43.157732 1708 leader_election.cc:341] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.158160 1708 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.159296 1864 raft_consensus.cc:2726] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.162038 1653 tablet_service.cc:1467] Processing CreateTablet for tablet 246c3d015ec242eca8bfaf5a076692b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260619 21:15:43.162514 1858 tablet_bootstrap.cc:492] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.163399 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246c3d015ec242eca8bfaf5a076692b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.166128 1800 tablet_service.cc:1467] Processing CreateTablet for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260619 21:15:43.167434 1859 raft_consensus.cc:348] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.168033 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.168231 1859 raft_consensus.cc:374] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.168586 1859 raft_consensus.cc:729] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.173249 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b2adcb38244b60bdd7ae5d5642f641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.173501 1651 tablet_service.cc:1467] Processing CreateTablet for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260619 21:15:43.187881 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.191372 1655 tablet_service.cc:1467] Processing CreateTablet for tablet b249d6b40f6943e3bb7235b7df99e1b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260619 21:15:43.192427 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b249d6b40f6943e3bb7235b7df99e1b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260619 21:15:43.185514 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 64011.5us
I20260619 21:15:43.183751 1863 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.195819 1863 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.196605 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:43.196906 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:43.197244 1743 raft_consensus.cc:3037] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:43.198351 1635 leader_election.cc:341] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.202133 1654 tablet_service.cc:1467] Processing CreateTablet for tablet d80394dabe604f8898e7b24149fc624f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260619 21:15:43.202893 1658 tablet_service.cc:1467] Processing CreateTablet for tablet 692d38d1f19c468eae7077dde218a62f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260619 21:15:43.162041 1652 tablet_service.cc:1467] Processing CreateTablet for tablet cfa69d158f2b471e8638824d3d9c757b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260619 21:15:43.187676 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e50dfc7ff65482ab6eb4b18b5ae06bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.203471 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80394dabe604f8898e7b24149fc624f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.229049 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692d38d1f19c468eae7077dde218a62f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.211184 1805 tablet_service.cc:1467] Processing CreateTablet for tablet 246c3d015ec242eca8bfaf5a076692b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260619 21:15:43.231124 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246c3d015ec242eca8bfaf5a076692b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.214543 1860 tablet_bootstrap.cc:492] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.236682 1860 ts_tablet_manager.cc:1397] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.200s user 0.019s sys 0.007s
I20260619 21:15:43.235920 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa69d158f2b471e8638824d3d9c757b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.214797 1859 consensus_queue.cc:260] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.216811 1743 raft_consensus.cc:2454] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:43.252166 1634 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.219252 1806 tablet_service.cc:1467] Processing CreateTablet for tablet b249d6b40f6943e3bb7235b7df99e1b1 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260619 21:15:43.254091 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b249d6b40f6943e3bb7235b7df99e1b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.257117 1863 raft_consensus.cc:2781] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:43.258518 1863 raft_consensus.cc:686] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:43.261458 1657 tablet_service.cc:1467] Processing CreateTablet for tablet bb9b500bf9a24148b2fbcd762b549521 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260619 21:15:43.261392 1863 consensus_queue.cc:237] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.260142 1860 raft_consensus.cc:348] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.262499 1860 raft_consensus.cc:374] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.262837 1860 raft_consensus.cc:729] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.263993 1860 consensus_queue.cc:260] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.266278 1860 ts_tablet_manager.cc:1428] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.029s user 0.003s sys 0.003s
I20260619 21:15:43.220165 1799 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 35) took 1056 ms. Trace:
I20260619 21:15:43.270226 1799 rpcz_store.cc:276] 0619 21:15:42.163357 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:42.622870 (+459513us) service_pool.cc:227] Handling call
0619 21:15:42.664007 (+ 41137us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5ec6c 00007f719bb5ebd6 000000000034aea9 00007f71a398aa30 00007f71a3a6cd3c 00007f71a3a4a1e3 00007f71a39fb869 00007f719cffdc44 00007f719cffdbd2 00007f719cffdb01
0619 21:15:42.731783 (+ 67776us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:15:42.732150 (+ 367us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:15:43.092762 (+360612us) tablet_metadata.cc:641] Metadata flushed
0619 21:15:43.176862 (+ 84100us) tablet_replica.cc:205] Creating consensus instance
0619 21:15:43.220079 (+ 43217us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8394368}
I20260619 21:15:43.273617 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb9b500bf9a24148b2fbcd762b549521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.273397 1799 tablet_service.cc:1467] Processing CreateTablet for tablet 692d38d1f19c468eae7077dde218a62f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260619 21:15:43.275044 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692d38d1f19c468eae7077dde218a62f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.279296 1807 tablet_service.cc:1467] Processing CreateTablet for tablet bb9b500bf9a24148b2fbcd762b549521 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260619 21:15:43.280527 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb9b500bf9a24148b2fbcd762b549521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.219700 1858 tablet_bootstrap.cc:654] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.242269 1859 ts_tablet_manager.cc:1428] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.104s user 0.008s sys 0.000s
I20260619 21:15:43.298028 1859 tablet_bootstrap.cc:492] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.301744 1804 tablet_service.cc:1467] Processing CreateTablet for tablet d80394dabe604f8898e7b24149fc624f (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260619 21:15:43.303088 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80394dabe604f8898e7b24149fc624f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.212181 1801 tablet_service.cc:1467] Processing CreateTablet for tablet cfa69d158f2b471e8638824d3d9c757b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260619 21:15:43.315554 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa69d158f2b471e8638824d3d9c757b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:15:43.317225 1859 tablet_bootstrap.cc:654] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.313256 1860 tablet_bootstrap.cc:492] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.326042 1858 tablet_bootstrap.cc:492] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.326565 1858 ts_tablet_manager.cc:1397] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.164s user 0.009s sys 0.005s
I20260619 21:15:43.329470 1858 raft_consensus.cc:348] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.330164 1858 raft_consensus.cc:374] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.330470 1858 raft_consensus.cc:729] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.331058 1858 consensus_queue.cc:260] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.333086 1858 ts_tablet_manager.cc:1428] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260619 21:15:43.334328 1858 tablet_bootstrap.cc:492] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.339980 1858 tablet_bootstrap.cc:654] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.321019 1567 catalog_manager.cc:5495] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:43.354324 1858 tablet_bootstrap.cc:492] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.354837 1858 ts_tablet_manager.cc:1397] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.016s
I20260619 21:15:43.351063 1860 tablet_bootstrap.cc:654] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.357595 1858 raft_consensus.cc:348] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.359176 1858 raft_consensus.cc:374] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.360318 1858 raft_consensus.cc:729] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.361685 1858 consensus_queue.cc:260] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.370095 1859 tablet_bootstrap.cc:492] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.370959 1859 ts_tablet_manager.cc:1397] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.073s user 0.007s sys 0.007s
I20260619 21:15:43.372601 1858 ts_tablet_manager.cc:1428] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.017s user 0.008s sys 0.001s
I20260619 21:15:43.374441 1859 raft_consensus.cc:348] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.375516 1859 raft_consensus.cc:374] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.375864 1859 raft_consensus.cc:729] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.378052 1858 tablet_bootstrap.cc:492] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.382020 1859 consensus_queue.cc:260] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.387446 1858 tablet_bootstrap.cc:654] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.389199 1859 ts_tablet_manager.cc:1428] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.018s user 0.009s sys 0.000s
I20260619 21:15:43.390257 1859 tablet_bootstrap.cc:492] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.409946 1859 tablet_bootstrap.cc:654] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.410135 1858 tablet_bootstrap.cc:492] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.410844 1858 ts_tablet_manager.cc:1397] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.004s
I20260619 21:15:43.413237 1860 tablet_bootstrap.cc:492] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.414000 1860 ts_tablet_manager.cc:1397] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.145s user 0.016s sys 0.007s
I20260619 21:15:43.421932 1858 raft_consensus.cc:348] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.422133 1860 raft_consensus.cc:348] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.422803 1858 raft_consensus.cc:374] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.422873 1860 raft_consensus.cc:374] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.423198 1860 raft_consensus.cc:729] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.423166 1858 raft_consensus.cc:729] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.424036 1860 consensus_queue.cc:260] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.424135 1858 consensus_queue.cc:260] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.426614 1858 ts_tablet_manager.cc:1428] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260619 21:15:43.427611 1858 tablet_bootstrap.cc:492] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.434705 1860 ts_tablet_manager.cc:1428] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.020s user 0.001s sys 0.005s
I20260619 21:15:43.436012 1860 tablet_bootstrap.cc:492] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.441129 1859 tablet_bootstrap.cc:492] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.441639 1859 ts_tablet_manager.cc:1397] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.052s user 0.022s sys 0.014s
I20260619 21:15:43.442718 1860 tablet_bootstrap.cc:654] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.444293 1859 raft_consensus.cc:348] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.445063 1859 raft_consensus.cc:374] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.445329 1859 raft_consensus.cc:729] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.445945 1858 tablet_bootstrap.cc:654] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.446108 1859 consensus_queue.cc:260] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.448678 1859 ts_tablet_manager.cc:1428] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 21:15:43.449851 1859 tablet_bootstrap.cc:492] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.455950 1859 tablet_bootstrap.cc:654] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.457819 1858 tablet_bootstrap.cc:492] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.458369 1858 ts_tablet_manager.cc:1397] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.006s
I20260619 21:15:43.463625 1863 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.464133 1863 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.466130 1863 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.465739 1858 raft_consensus.cc:348] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.467527 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.467645 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.468194 1818 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:43.468329 1743 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:43.469806 1635 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:43.470854 1863 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.466967 1858 raft_consensus.cc:374] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.471262 1863 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.471621 1863 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.471339 1858 raft_consensus.cc:729] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.472400 1858 consensus_queue.cc:260] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.474612 1860 tablet_bootstrap.cc:492] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.475100 1860 ts_tablet_manager.cc:1397] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.010s
I20260619 21:15:43.477952 1860 raft_consensus.cc:348] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.478667 1860 raft_consensus.cc:374] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.478991 1860 raft_consensus.cc:729] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.479830 1860 consensus_queue.cc:260] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.481302 1863 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.482389 1860 ts_tablet_manager.cc:1428] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260619 21:15:43.484905 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:43.485306 1858 ts_tablet_manager.cc:1428] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.027s user 0.009s sys 0.000s
I20260619 21:15:43.485692 1743 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.487991 1860 tablet_bootstrap.cc:492] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.492588 1858 tablet_bootstrap.cc:492] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.492686 1883 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.493351 1883 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.497336 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.497996 1883 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:43.499446 1635 leader_election.cc:341] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.499938 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.501153 1858 tablet_bootstrap.cc:654] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:43.501277 1634 leader_election.cc:341] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.501768 1634 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:43.502097 1859 tablet_bootstrap.cc:492] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.502671 1859 ts_tablet_manager.cc:1397] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.053s user 0.020s sys 0.016s
I20260619 21:15:43.502871 1883 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.504416 1860 tablet_bootstrap.cc:654] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.505584 1863 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.506630 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:43.506609 1859 raft_consensus.cc:348] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.507320 1818 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.507754 1859 raft_consensus.cc:374] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.508159 1859 raft_consensus.cc:729] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.509239 1859 consensus_queue.cc:260] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.511677 1859 ts_tablet_manager.cc:1428] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260619 21:15:43.512760 1859 tablet_bootstrap.cc:492] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.518687 1860 tablet_bootstrap.cc:492] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.519235 1859 tablet_bootstrap.cc:654] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.519400 1860 ts_tablet_manager.cc:1397] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.011s
I20260619 21:15:43.523074 1743 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:43.523388 1860 raft_consensus.cc:348] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.524678 1860 raft_consensus.cc:374] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.524636 1634 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:43.525871 1883 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:43.527089 1818 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:43.526741 1860 raft_consensus.cc:729] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.533495 1883 raft_consensus.cc:686] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:43.534755 1883 consensus_queue.cc:237] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.538208 1860 consensus_queue.cc:260] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.539963 1858 tablet_bootstrap.cc:492] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.540486 1859 tablet_bootstrap.cc:492] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.540472 1858 ts_tablet_manager.cc:1397] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.048s user 0.017s sys 0.023s
I20260619 21:15:43.541133 1859 ts_tablet_manager.cc:1397] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.015s
I20260619 21:15:43.543460 1859 raft_consensus.cc:348] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.544135 1859 raft_consensus.cc:374] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.543838 1858 raft_consensus.cc:348] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.544248 1860 ts_tablet_manager.cc:1428] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.024s user 0.010s sys 0.000s
I20260619 21:15:43.544438 1859 raft_consensus.cc:729] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.544579 1858 raft_consensus.cc:374] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.544889 1858 raft_consensus.cc:729] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.545477 1860 tablet_bootstrap.cc:492] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.545159 1859 consensus_queue.cc:260] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.545648 1858 consensus_queue.cc:260] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.547307 1859 ts_tablet_manager.cc:1428] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260619 21:15:43.548066 1859 tablet_bootstrap.cc:492] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.552711 1858 ts_tablet_manager.cc:1428] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260619 21:15:43.553035 1859 tablet_bootstrap.cc:654] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.553823 1858 tablet_bootstrap.cc:492] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.561792 1860 tablet_bootstrap.cc:654] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.564589 1858 tablet_bootstrap.cc:654] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.565011 1885 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.565564 1885 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.567626 1859 tablet_bootstrap.cc:492] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.568213 1859 ts_tablet_manager.cc:1397] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.011s
I20260619 21:15:43.569907 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.570780 1669 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:43.571507 1859 raft_consensus.cc:348] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.572451 1859 raft_consensus.cc:374] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.572770 1859 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.573583 1859 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.575400 1782 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:43.576359 1859 ts_tablet_manager.cc:1428] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260619 21:15:43.576609 1875 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.577127 1875 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.577533 1875 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.577631 1859 tablet_bootstrap.cc:492] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.581281 1885 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.573951 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:43.583480 1783 leader_election.cc:341] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.586330 1859 tablet_bootstrap.cc:654] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.563632 1566 catalog_manager.cc:5495] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:43.606310 1875 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.613315 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:43.613997 1669 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.631243 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:15:43.633543 1783 leader_election.cc:341] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.634615 1875 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.638561 1858 tablet_bootstrap.cc:492] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.639045 1858 ts_tablet_manager.cc:1397] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.085s user 0.035s sys 0.043s
I20260619 21:15:43.639631 1669 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:43.639700 1860 tablet_bootstrap.cc:492] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.640312 1860 ts_tablet_manager.cc:1397] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.095s user 0.032s sys 0.042s
I20260619 21:15:43.640825 1782 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:43.641435 1858 raft_consensus.cc:348] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.642071 1875 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:43.642158 1858 raft_consensus.cc:374] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.642556 1858 raft_consensus.cc:729] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.642815 1875 raft_consensus.cc:686] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:43.643882 1859 tablet_bootstrap.cc:492] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.643355 1858 consensus_queue.cc:260] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.643702 1875 consensus_queue.cc:237] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.644447 1859 ts_tablet_manager.cc:1397] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.067s user 0.020s sys 0.030s
I20260619 21:15:43.644737 1860 raft_consensus.cc:348] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.645470 1860 raft_consensus.cc:374] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.645740 1860 raft_consensus.cc:729] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.646162 1858 ts_tablet_manager.cc:1428] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260619 21:15:43.646417 1860 consensus_queue.cc:260] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.646908 1859 raft_consensus.cc:348] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.647699 1858 tablet_bootstrap.cc:492] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.647679 1859 raft_consensus.cc:374] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.648114 1859 raft_consensus.cc:729] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.648835 1860 ts_tablet_manager.cc:1428] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260619 21:15:43.648758 1859 consensus_queue.cc:260] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.649186 1885 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.649736 1885 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.650966 1859 ts_tablet_manager.cc:1428] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 21:15:43.651674 1885 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.652251 1859 tablet_bootstrap.cc:492] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.653282 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.653669 1858 tablet_bootstrap.cc:654] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.654374 1860 tablet_bootstrap.cc:492] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
W20260619 21:15:43.657666 1783 leader_election.cc:341] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.658010 1859 tablet_bootstrap.cc:654] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.659477 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.660157 1669 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:43.660540 1860 tablet_bootstrap.cc:654] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.661489 1782 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:43.663154 1885 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.663940 1885 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.664670 1885 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.665495 1858 tablet_bootstrap.cc:492] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.666070 1858 ts_tablet_manager.cc:1397] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.006s
I20260619 21:15:43.669270 1858 raft_consensus.cc:348] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.670161 1858 raft_consensus.cc:374] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.670480 1858 raft_consensus.cc:729] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.671478 1858 consensus_queue.cc:260] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.675055 1859 tablet_bootstrap.cc:492] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.675174 1858 ts_tablet_manager.cc:1428] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260619 21:15:43.675527 1859 ts_tablet_manager.cc:1397] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.023s user 0.015s sys 0.006s
I20260619 21:15:43.676055 1858 tablet_bootstrap.cc:492] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.678472 1859 raft_consensus.cc:348] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.679211 1859 raft_consensus.cc:374] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.679488 1859 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.680114 1859 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.681836 1858 tablet_bootstrap.cc:654] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.682163 1859 ts_tablet_manager.cc:1428] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 21:15:43.683208 1859 tablet_bootstrap.cc:492] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.690042 1859 tablet_bootstrap.cc:654] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.694115 1858 tablet_bootstrap.cc:492] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.694738 1858 ts_tablet_manager.cc:1397] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.003s
I20260619 21:15:43.696633 1864 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.697297 1864 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.701313 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.702315 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.703027 1669 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:43.703043 1864 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:43.704185 1710 leader_election.cc:341] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.705235 1708 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.707024 1864 raft_consensus.cc:2781] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.707557 1864 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.708065 1864 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.701344 1885 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.712438 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:43.713094 1669 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.720103 1858 raft_consensus.cc:348] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.727653 1858 raft_consensus.cc:374] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.729321 1858 raft_consensus.cc:729] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.731606 1669 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:43.732218 1885 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.734004 1858 consensus_queue.cc:260] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.734544 1782 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:43.735615 1885 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:43.736156 1860 tablet_bootstrap.cc:492] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.736336 1885 raft_consensus.cc:686] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:43.736744 1860 ts_tablet_manager.cc:1397] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.087s user 0.022s sys 0.018s
I20260619 21:15:43.737630 1917 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.737363 1885 consensus_queue.cc:237] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.738246 1917 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.726778 1566 catalog_manager.cc:5495] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:43.739989 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:43.739900 1860 raft_consensus.cc:348] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.740590 1860 raft_consensus.cc:374] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.740839 1860 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.740871 1864 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.741765 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:43.745421 1708 leader_election.cc:341] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.748351 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.748803 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:43.749450 1669 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:43.750681 1710 leader_election.cc:341] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.741665 1860 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:43.742661 1783 leader_election.cc:341] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.751268 1710 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.752889 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:43.753882 1875 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.754594 1860 ts_tablet_manager.cc:1428] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.017s user 0.004s sys 0.003s
W20260619 21:15:43.754514 1710 leader_election.cc:341] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.755781 1860 tablet_bootstrap.cc:492] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.756153 1916 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.743647 1858 ts_tablet_manager.cc:1428] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.049s user 0.007s sys 0.003s
I20260619 21:15:43.757671 1859 tablet_bootstrap.cc:492] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.757939 1858 tablet_bootstrap.cc:492] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.758117 1859 ts_tablet_manager.cc:1397] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.075s user 0.035s sys 0.023s
I20260619 21:15:43.754510 1875 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.761118 1864 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.761615 1859 raft_consensus.cc:348] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.762549 1859 raft_consensus.cc:374] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.762878 1859 raft_consensus.cc:729] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.744184 1917 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.768132 1860 tablet_bootstrap.cc:654] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.769274 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.770598 1668 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:43.771083 1875 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.765323 1859 consensus_queue.cc:260] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.772395 1782 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:43.773803 1875 raft_consensus.cc:2781] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.774363 1875 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.774921 1859 ts_tablet_manager.cc:1428] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260619 21:15:43.774917 1875 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.775625 1669 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:43.777066 1859 tablet_bootstrap.cc:492] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:43.778842 1708 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.780694 1917 raft_consensus.cc:2781] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:43.781332 1917 raft_consensus.cc:686] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:43.781672 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.782490 1743 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:43.782620 1917 consensus_queue.cc:237] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.791549 1864 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.792177 1864 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.792055 1875 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.797219 1864 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.798058 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.799685 1858 tablet_bootstrap.cc:654] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.800894 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.802300 1669 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:43.802943 1885 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.803699 1885 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.806286 1708 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.813908 1864 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.814043 1566 catalog_manager.cc:5495] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:43.807437 1875 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.807670 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:43.818501 1668 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.818889 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.818975 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:43.819862 1669 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:43.820050 1744 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:43.809512 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:43.821233 1743 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.821425 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:43.822971 1875 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.823472 1875 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.824004 1875 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.812516 1859 tablet_bootstrap.cc:654] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.830114 1885 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.838696 1864 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.843895 1743 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:43.845764 1567 catalog_manager.cc:5495] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:43.847891 1668 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:43.850123 1783 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:43.853684 1885 raft_consensus.cc:2781] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
W20260619 21:15:43.857054 1635 consensus_peers.cc:487] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:43.859185 1864 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.859616 1864 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.861338 1864 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:43.863235 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.863976 1860 tablet_bootstrap.cc:492] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:43.864867 1860 ts_tablet_manager.cc:1397] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.109s user 0.005s sys 0.023s
I20260619 21:15:43.862474 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:43.865338 1710 leader_election.cc:341] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.866346 1885 raft_consensus.cc:686] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:15:43.866639 1708 leader_election.cc:341] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:43.867259 1708 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:43.867689 1885 consensus_queue.cc:237] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.870543 1864 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:43.874553 1860 raft_consensus.cc:348] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.875216 1860 raft_consensus.cc:374] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.875475 1860 raft_consensus.cc:729] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.875497 1858 tablet_bootstrap.cc:492] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:43.876242 1858 ts_tablet_manager.cc:1397] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.119s user 0.005s sys 0.013s
I20260619 21:15:43.876152 1860 consensus_queue.cc:260] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.876597 1875 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.877826 1860 ts_tablet_manager.cc:1428] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.013s user 0.005s sys 0.005s
I20260619 21:15:43.878502 1860 tablet_bootstrap.cc:492] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:43.880218 1858 raft_consensus.cc:348] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.881984 1858 raft_consensus.cc:374] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.882313 1858 raft_consensus.cc:729] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.883023 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:43.883555 1859 tablet_bootstrap.cc:492] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:43.883064 1858 consensus_queue.cc:260] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.884045 1859 ts_tablet_manager.cc:1397] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.107s user 0.020s sys 0.003s
I20260619 21:15:43.883937 1668 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.886200 1875 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:43.887820 1858 ts_tablet_manager.cc:1428] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260619 21:15:43.890862 1859 raft_consensus.cc:348] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.891685 1859 raft_consensus.cc:374] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:43.892048 1859 raft_consensus.cc:729] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:43.892604 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:43.893020 1858 tablet_bootstrap.cc:492] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:43.893446 1744 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.913110 1864 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.917937 1864 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.919440 1668 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:43.919811 1744 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:43.920506 1860 tablet_bootstrap.cc:654] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:43.922035 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:43.923511 1875 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:43.923723 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.924345 1875 raft_consensus.cc:686] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:43.925966 1875 consensus_queue.cc:237] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.927102 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:43.928005 1818 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:43.929468 1710 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:43.924724 1669 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:43.930506 1917 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:43.977602 1917 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:43.931803 1567 catalog_manager.cc:5495] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:43.978112 1917 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.937649 1916 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:43.980790 1858 tablet_bootstrap.cc:654] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.012174 1917 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.980737 1916 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.030658 1567 catalog_manager.cc:5495] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:43.989845 1859 consensus_queue.cc:260] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:43.921373 1864 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.033802 1883 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.034562 1883 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.037142 1916 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.038756 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:43.990674 1903 consensus_queue.cc:1035] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260619 21:15:44.033855 1917 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:44.042349 1708 leader_election.cc:341] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.047363 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:44.048198 1818 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:43.998329 1863 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.050189 1863 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.054663 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:44.055454 1669 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.055217 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.056273 1816 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:44.053187 1863 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:43.989414 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 91556.2us
I20260619 21:15:44.064046 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.063006 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:15:44.067385 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:44.069262 1883 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.070079 1883 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:44.070578 1883 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:44.071053 1883 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:44.073963 1710 leader_election.cc:341] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.074694 1710 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:44.076123 1916 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.076930 1864 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.080771 1875 consensus_queue.cc:1035] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.077925 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.081339 1864 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.081192 1863 consensus_queue.cc:1035] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.079041 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.079038 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.083880 1818 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:44.087921 1742 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:44.088829 1817 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
W20260619 21:15:44.090423 1634 leader_election.cc:341] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.095288 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:44.095901 1635 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:44.096603 1667 raft_consensus.cc:2379] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:44.097502 1710 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:44.100749 1916 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:44.101483 1916 raft_consensus.cc:686] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:44.102950 1916 consensus_queue.cc:237] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.106233 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.097061 1864 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.107297 1926 consensus_queue.cc:1035] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.107777 1859 ts_tablet_manager.cc:1428] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.223s user 0.013s sys 0.009s
I20260619 21:15:44.114831 1859 tablet_bootstrap.cc:492] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:44.121917 1883 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.101383 1903 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:44.125589 1859 tablet_bootstrap.cc:654] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.106235 1863 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.126668 1903 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:44.134651 1883 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.135139 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:44.138022 1931 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.131440 1903 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.140554 1858 tablet_bootstrap.cc:492] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:44.142037 1858 ts_tablet_manager.cc:1397] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.249s user 0.024s sys 0.012s
I20260619 21:15:44.128441 1863 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.139786 1864 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.145238 1860 tablet_bootstrap.cc:492] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:44.145639 1860 ts_tablet_manager.cc:1397] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.267s user 0.006s sys 0.010s
I20260619 21:15:44.131796 1710 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:44.151041 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:44.151773 1817 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:44.153599 1634 leader_election.cc:341] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.147332 1864 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.156304 1864 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.158912 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.159747 1818 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:44.161453 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:44.163272 1863 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.163407 1710 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:44.164669 1864 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:44.165271 1864 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:44.166188 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.141142 1931 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.167632 1667 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:44.167830 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.166200 1864 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.170713 1858 raft_consensus.cc:348] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.172784 1858 raft_consensus.cc:374] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.173156 1858 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.173424 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.174434 1858 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.150204 1916 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:44.173722 1635 leader_election.cc:341] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.179960 1931 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.180636 1860 raft_consensus.cc:348] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.184585 1860 raft_consensus.cc:374] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.184991 1860 raft_consensus.cc:729] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.185776 1860 consensus_queue.cc:260] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.188381 1860 ts_tablet_manager.cc:1428] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.043s user 0.007s sys 0.000s
W20260619 21:15:44.188886 1634 leader_election.cc:341] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.194290 1634 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:44.195489 1931 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.189675 1916 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.186609 1669 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:44.201534 1567 catalog_manager.cc:5495] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:44.204782 1858 ts_tablet_manager.cc:1428] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.062s user 0.008s sys 0.000s
I20260619 21:15:44.207701 1860 tablet_bootstrap.cc:492] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:44.210465 1858 tablet_bootstrap.cc:492] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:44.213698 1875 consensus_queue.cc:1035] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.215624 1817 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:44.223601 1864 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.228071 1740 raft_consensus.cc:2379] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:15:44.230983 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 57330.4us
I20260619 21:15:44.231225 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:44.232053 1669 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.234193 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.235150 1818 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:44.235241 1903 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.258736 1885 consensus_queue.cc:1035] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.261574 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:44.262490 1816 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:44.262521 1710 consensus_peers.cc:487] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:44.261612 1864 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.253043 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:44.266387 1883 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:44.267052 1883 raft_consensus.cc:686] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:44.268342 1883 consensus_queue.cc:237] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.270721 1864 consensus_queue.cc:1035] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.282673 1859 tablet_bootstrap.cc:492] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:44.285252 1669 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:44.288264 1859 ts_tablet_manager.cc:1397] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.174s user 0.012s sys 0.022s
I20260619 21:15:44.289814 1708 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:44.291384 1864 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:44.293262 1864 raft_consensus.cc:686] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:44.296567 1864 consensus_queue.cc:237] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.299145 1860 tablet_bootstrap.cc:654] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.313004 1635 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:44.351222 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 50241.7us
I20260619 21:15:44.353148 1926 raft_consensus.cc:2741] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260619 21:15:44.354205 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:44.355098 1818 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.356630 1816 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:44.361428 1858 tablet_bootstrap.cc:654] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.338209 1903 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.367825 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:15:44.375242 1941 log.cc:924] Time spent T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.053s user 0.000s sys 0.001s
I20260619 21:15:44.373208 1740 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.399032 1885 consensus_queue.cc:1035] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.400074 1859 raft_consensus.cc:348] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.402338 1859 raft_consensus.cc:374] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.402704 1859 raft_consensus.cc:729] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.411150 1859 consensus_queue.cc:260] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:44.408371 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 48921.5us
I20260619 21:15:44.419875 1859 ts_tablet_manager.cc:1428] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.128s user 0.007s sys 0.001s
I20260619 21:15:44.421097 1859 tablet_bootstrap.cc:492] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:44.423243 1909 consensus_queue.cc:1035] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260619 21:15:44.418248 1864 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.426249 1885 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.432179 1885 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.429464 1909 consensus_queue.cc:1035] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260619 21:15:44.437098 1885 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:44.438301 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.430336 1864 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.446240 1860 tablet_bootstrap.cc:492] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:44.446645 1860 ts_tablet_manager.cc:1397] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.239s user 0.016s sys 0.022s
I20260619 21:15:44.450945 1858 tablet_bootstrap.cc:492] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:44.451473 1858 ts_tablet_manager.cc:1397] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.244s user 0.022s sys 0.003s
I20260619 21:15:44.454298 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:44.454942 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:44.463845 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.465225 1817 raft_consensus.cc:2379] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:44.465691 1858 raft_consensus.cc:348] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.467123 1858 raft_consensus.cc:374] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.467545 1858 raft_consensus.cc:729] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.468472 1858 consensus_queue.cc:260] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.469758 1864 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.455406 1909 consensus_queue.cc:1035] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260619 21:15:44.468183 1710 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:44.441169 1564 catalog_manager.cc:5495] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:44.440259 1818 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:44.479132 1859 tablet_bootstrap.cc:654] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.478237 1783 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:44.479395 1635 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:44.473807 1864 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.482152 1926 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:44.482936 1926 raft_consensus.cc:686] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:44.484973 1926 consensus_queue.cc:237] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.484128 1909 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.502996 1903 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.503813 1903 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.508669 1858 ts_tablet_manager.cc:1428] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.057s user 0.004s sys 0.008s
I20260619 21:15:44.510038 1858 tablet_bootstrap.cc:492] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:44.508564 1903 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.514225 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:44.516568 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.517094 1818 raft_consensus.cc:3037] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:44.516568 1635 leader_election.cc:341] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.520750 1909 consensus_queue.cc:1035] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260619 21:15:44.528111 1634 leader_election.cc:341] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.528398 1860 raft_consensus.cc:348] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.535606 1917 consensus_queue.cc:1035] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.530736 1634 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:44.541045 1903 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.535631 1860 raft_consensus.cc:374] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.545150 1860 raft_consensus.cc:729] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.562479 1859 tablet_bootstrap.cc:492] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:44.560480 1860 consensus_queue.cc:260] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.563006 1859 ts_tablet_manager.cc:1397] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.142s user 0.011s sys 0.023s
I20260619 21:15:44.570252 1858 tablet_bootstrap.cc:654] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.580416 1875 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.581425 1875 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.585886 1875 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:44.591392 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:44.594875 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.599527 1864 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.584204 1740 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:44.595609 1859 raft_consensus.cc:348] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.601676 1859 raft_consensus.cc:374] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.602504 1859 raft_consensus.cc:729] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.604909 1860 ts_tablet_manager.cc:1428] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.158s user 0.012s sys 0.010s
I20260619 21:15:44.603730 1859 consensus_queue.cc:260] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.608878 1860 tablet_bootstrap.cc:492] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:44.606861 1783 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:44.611900 1909 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.618738 1859 ts_tablet_manager.cc:1428] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.055s user 0.004s sys 0.004s
I20260619 21:15:44.620042 1859 tablet_bootstrap.cc:492] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:44.627076 1917 consensus_queue.cc:1035] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.630229 1565 catalog_manager.cc:5495] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:44.665822 1859 tablet_bootstrap.cc:654] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.681473 1860 tablet_bootstrap.cc:654] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.682155 1858 tablet_bootstrap.cc:492] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:44.683040 1858 ts_tablet_manager.cc:1397] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.173s user 0.010s sys 0.014s
I20260619 21:15:44.711133 1858 raft_consensus.cc:348] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.713621 1858 raft_consensus.cc:374] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.713959 1858 raft_consensus.cc:729] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.714768 1858 consensus_queue.cc:260] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.716234 1860 tablet_bootstrap.cc:492] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:44.716727 1860 ts_tablet_manager.cc:1397] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.111s user 0.016s sys 0.004s
I20260619 21:15:44.717865 1858 ts_tablet_manager.cc:1428] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.035s user 0.013s sys 0.015s
I20260619 21:15:44.719177 1858 tablet_bootstrap.cc:492] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:44.719902 1860 raft_consensus.cc:348] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.720479 1860 raft_consensus.cc:374] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.720741 1860 raft_consensus.cc:729] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.721410 1860 consensus_queue.cc:260] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.723181 1860 ts_tablet_manager.cc:1428] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260619 21:15:44.724159 1860 tablet_bootstrap.cc:492] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:44.728112 1858 tablet_bootstrap.cc:654] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.728876 1859 tablet_bootstrap.cc:492] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:44.731168 1859 ts_tablet_manager.cc:1397] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.111s user 0.014s sys 0.019s
I20260619 21:15:44.733484 1860 tablet_bootstrap.cc:654] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.735039 1818 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.742332 1567 catalog_manager.cc:5495] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:44.750665 1864 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.745970 1859 raft_consensus.cc:348] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.752753 1859 raft_consensus.cc:374] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.753208 1859 raft_consensus.cc:729] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.754081 1859 consensus_queue.cc:260] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.758371 1859 ts_tablet_manager.cc:1428] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20260619 21:15:44.759558 1859 tablet_bootstrap.cc:492] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:44.768929 1858 tablet_bootstrap.cc:492] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:44.769534 1858 ts_tablet_manager.cc:1397] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.051s user 0.006s sys 0.020s
I20260619 21:15:44.771395 1859 tablet_bootstrap.cc:654] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.772832 1858 raft_consensus.cc:348] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.773691 1858 raft_consensus.cc:374] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.776736 1858 raft_consensus.cc:729] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.786795 1858 consensus_queue.cc:260] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.793437 1858 ts_tablet_manager.cc:1428] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.024s user 0.009s sys 0.000s
I20260619 21:15:44.794659 1926 consensus_queue.cc:1035] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.804749 1858 tablet_bootstrap.cc:492] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:44.827134 1860 tablet_bootstrap.cc:492] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:44.827679 1860 ts_tablet_manager.cc:1397] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.104s user 0.021s sys 0.020s
I20260619 21:15:44.823307 1903 consensus_queue.cc:1035] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260619 21:15:44.838804 1860 raft_consensus.cc:348] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.843616 1864 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:44.843623 1860 raft_consensus.cc:374] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.848035 1860 raft_consensus.cc:729] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.843201 1859 tablet_bootstrap.cc:492] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:44.848908 1859 ts_tablet_manager.cc:1397] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.090s user 0.024s sys 0.013s
I20260619 21:15:44.850560 1860 consensus_queue.cc:260] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.863711 1860 ts_tablet_manager.cc:1428] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.036s user 0.003s sys 0.007s
I20260619 21:15:44.880187 1860 tablet_bootstrap.cc:492] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:44.874842 1859 raft_consensus.cc:348] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.883145 1859 raft_consensus.cc:374] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.883464 1859 raft_consensus.cc:729] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.887337 1859 consensus_queue.cc:260] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.895152 1858 tablet_bootstrap.cc:654] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.897740 1931 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.898437 1931 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.904237 1931 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.908385 1859 ts_tablet_manager.cc:1428] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.059s user 0.004s sys 0.003s
I20260619 21:15:44.910048 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:44.916572 1635 leader_election.cc:341] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.917347 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.918467 1740 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:44.920786 1859 tablet_bootstrap.cc:492] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:44.921249 1634 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:44.922789 1931 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:44.923367 1931 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:44.923933 1931 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:44.927862 1917 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.930024 1917 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.933120 1917 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.934746 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:44.936458 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:44.936764 1708 leader_election.cc:341] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:44.938594 1710 leader_election.cc:341] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.939232 1710 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:44.940322 1917 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.942480 1858 tablet_bootstrap.cc:492] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:44.943289 1858 ts_tablet_manager.cc:1397] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.139s user 0.007s sys 0.013s
I20260619 21:15:44.945362 1860 tablet_bootstrap.cc:654] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.946983 1859 tablet_bootstrap.cc:654] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.947460 1883 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:44.947326 1858 raft_consensus.cc:348] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.948797 1858 raft_consensus.cc:374] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.949188 1858 raft_consensus.cc:729] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.948486 1883 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.950255 1858 consensus_queue.cc:260] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.952080 1883 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.954499 1858 ts_tablet_manager.cc:1428] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260619 21:15:44.954269 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:44.956269 1635 leader_election.cc:341] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.956058 1931 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.957548 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:44.958124 1858 tablet_bootstrap.cc:492] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
W20260619 21:15:44.961524 1634 leader_election.cc:341] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.962157 1634 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:44.963994 1883 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:44.964679 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:44.965898 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:44.966694 1740 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:44.967175 1635 leader_election.cc:341] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:44.967989 1931 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:44.969338 1858 tablet_bootstrap.cc:654] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:44.977365 1740 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:44.978091 1858 tablet_bootstrap.cc:492] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:44.978596 1858 ts_tablet_manager.cc:1397] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.008s
I20260619 21:15:44.978842 1634 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:44.980016 1931 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:44.980880 1931 raft_consensus.cc:686] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:44.982338 1931 consensus_queue.cc:237] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.991711 1860 tablet_bootstrap.cc:492] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:44.992408 1860 ts_tablet_manager.cc:1397] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.112s user 0.013s sys 0.005s
I20260619 21:15:44.992337 1858 raft_consensus.cc:348] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.993475 1858 raft_consensus.cc:374] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:44.993819 1858 raft_consensus.cc:729] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:44.994885 1858 consensus_queue.cc:260] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:44.998485 1858 ts_tablet_manager.cc:1428] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.020s user 0.010s sys 0.002s
I20260619 21:15:44.999845 1858 tablet_bootstrap.cc:492] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:44.997962 1860 raft_consensus.cc:348] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.000717 1860 raft_consensus.cc:374] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.001188 1860 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.008260 1860 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.010316 1859 tablet_bootstrap.cc:492] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.010778 1858 tablet_bootstrap.cc:654] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.011015 1859 ts_tablet_manager.cc:1397] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.090s user 0.020s sys 0.023s
I20260619 21:15:45.011901 1860 ts_tablet_manager.cc:1428] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.019s user 0.010s sys 0.000s
I20260619 21:15:45.013872 1860 tablet_bootstrap.cc:492] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.014501 1859 raft_consensus.cc:348] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.015368 1859 raft_consensus.cc:374] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.015631 1859 raft_consensus.cc:729] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.016336 1859 consensus_queue.cc:260] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.019158 1859 ts_tablet_manager.cc:1428] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20260619 21:15:45.020639 1859 tablet_bootstrap.cc:492] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.020700 1864 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.021440 1864 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.023840 1958 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.025650 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.027654 1864 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:45.028442 1708 leader_election.cc:341] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.024659 1958 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.031167 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:45.033004 1710 leader_election.cc:341] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.039430 1710 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.036521 1859 tablet_bootstrap.cc:654] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.037181 1958 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.038055 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.038671 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.041234 1864 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.044358 1565 catalog_manager.cc:5495] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:45.052170 1931 consensus_queue.cc:1035] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:45.053124 1669 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
W20260619 21:15:45.053223 1783 leader_election.cc:341] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.062628 1782 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.065987 1958 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.079815 1958 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.083292 1958 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.073181 1858 tablet_bootstrap.cc:492] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.069824 1883 consensus_queue.cc:1035] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:45.084889 1858 ts_tablet_manager.cc:1397] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.085s user 0.017s sys 0.018s
I20260619 21:15:45.081787 1860 tablet_bootstrap.cc:654] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.091444 1946 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.092727 1946 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.098699 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.099594 1669 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:45.101147 1782 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:45.100719 1858 raft_consensus.cc:348] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.102299 1885 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.103034 1885 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.103551 1885 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.105130 1946 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.106307 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.102262 1858 raft_consensus.cc:374] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.114681 1858 raft_consensus.cc:729] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.125976 1931 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.126747 1931 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.134020 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.135783 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.136622 1741 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:45.138094 1634 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:45.139184 1883 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.139685 1883 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.140174 1883 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.117872 1858 consensus_queue.cc:260] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:45.142817 1635 leader_election.cc:341] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.144915 1743 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:45.148948 1946 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.150836 1946 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.149067 1864 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.152617 1885 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.153689 1864 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:45.155609 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:45.159020 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.161394 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.144014 1931 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.163911 1864 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:45.169430 1708 leader_election.cc:341] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:45.170374 1710 leader_election.cc:341] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.176465 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:45.177374 1669 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.177601 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.172199 1710 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.182762 1858 ts_tablet_manager.cc:1428] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.097s user 0.010s sys 0.001s
I20260619 21:15:45.184001 1858 tablet_bootstrap.cc:492] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
W20260619 21:15:45.185441 1782 leader_election.cc:341] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.186264 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.187157 1743 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.189857 1946 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.182076 1864 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.194145 1859 tablet_bootstrap.cc:492] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.183398 1885 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.194700 1859 ts_tablet_manager.cc:1397] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.174s user 0.020s sys 0.050s
I20260619 21:15:45.196760 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:45.199548 1783 leader_election.cc:341] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.200188 1783 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.202260 1860 tablet_bootstrap.cc:492] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.202755 1860 ts_tablet_manager.cc:1397] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.189s user 0.015s sys 0.021s
I20260619 21:15:45.203979 1885 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.206072 1859 raft_consensus.cc:348] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.206970 1859 raft_consensus.cc:374] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.207270 1859 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.208060 1859 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.208418 1860 raft_consensus.cc:348] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.209337 1860 raft_consensus.cc:374] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.207352 1864 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.209678 1860 raft_consensus.cc:729] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.210197 1864 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.199209 1958 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.213291 1859 ts_tablet_manager.cc:1428] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.018s user 0.010s sys 0.000s
I20260619 21:15:45.214643 1859 tablet_bootstrap.cc:492] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.214205 1883 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.210525 1860 consensus_queue.cc:260] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.219197 1903 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.219650 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.219818 1903 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.220407 1741 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.221428 1860 ts_tablet_manager.cc:1428] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.018s user 0.010s sys 0.001s
I20260619 21:15:45.222525 1860 tablet_bootstrap.cc:492] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.222985 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:45.223745 1667 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.226253 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.227275 1858 tablet_bootstrap.cc:654] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.227142 1742 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:45.232448 1903 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.236153 1864 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.236670 1958 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.236771 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.237630 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.239522 1743 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
W20260619 21:15:45.239715 1710 leader_election.cc:341] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.240211 1859 tablet_bootstrap.cc:654] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.240679 1883 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.241402 1783 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:45.242836 1860 tablet_bootstrap.cc:654] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.242997 1958 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.244155 1958 raft_consensus.cc:686] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:45.245638 1958 consensus_queue.cc:237] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.279094 1634 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:15:45.281097 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 49559.9us
I20260619 21:15:45.285465 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.287248 1669 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:45.287883 1741 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:45.290129 1859 tablet_bootstrap.cc:492] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.291169 1859 ts_tablet_manager.cc:1397] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.077s user 0.017s sys 0.002s
I20260619 21:15:45.292788 1858 tablet_bootstrap.cc:492] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
W20260619 21:15:45.292960 1783 leader_election.cc:341] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.293781 1858 ts_tablet_manager.cc:1397] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.110s user 0.016s sys 0.004s
I20260619 21:15:45.296916 1634 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:45.300580 1926 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.301204 1926 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.300910 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:45.301652 1926 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:45.302287 1708 leader_election.cc:341] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.303100 1708 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.304533 1917 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.298192 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.305114 1883 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.305789 1883 raft_consensus.cc:686] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:45.304553 1931 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.313153 1931 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.313153 1859 raft_consensus.cc:348] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.314328 1859 raft_consensus.cc:374] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.312271 1917 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.314643 1859 raft_consensus.cc:729] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.316612 1917 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:45.320262 1635 leader_election.cc:341] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.321539 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.307812 1883 consensus_queue.cc:237] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.322785 1917 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:45.327062 1710 leader_election.cc:341] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:45.329488 1635 leader_election.cc:341] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.330849 1931 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.331382 1859 consensus_queue.cc:260] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.337517 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.338171 1668 raft_consensus.cc:2454] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:45.338186 1926 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.340569 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.341419 1926 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:45.342285 1634 leader_election.cc:341] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.344444 1708 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.345167 1858 raft_consensus.cc:348] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.346207 1858 raft_consensus.cc:374] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.346540 1858 raft_consensus.cc:729] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.346906 1917 raft_consensus.cc:2781] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.347324 1917 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.347170 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.347713 1917 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.347404 1858 consensus_queue.cc:260] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.348130 1743 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.357731 1859 ts_tablet_manager.cc:1428] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.066s user 0.011s sys 0.000s
I20260619 21:15:45.357658 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.358826 1818 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:45.359669 1859 tablet_bootstrap.cc:492] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.363607 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.365651 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:45.368330 1667 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:45.376003 1858 ts_tablet_manager.cc:1428] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.082s user 0.008s sys 0.002s
I20260619 21:15:45.377502 1858 tablet_bootstrap.cc:492] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
W20260619 21:15:45.381609 1635 leader_election.cc:341] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.370719 1926 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.383066 1782 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.384979 1926 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.386953 1926 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.389699 1958 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.390862 1858 tablet_bootstrap.cc:654] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.390515 1958 raft_consensus.cc:686] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:45.392726 1958 consensus_queue.cc:237] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.396588 1859 tablet_bootstrap.cc:654] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.399372 1860 tablet_bootstrap.cc:492] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.399995 1860 ts_tablet_manager.cc:1397] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.178s user 0.014s sys 0.004s
I20260619 21:15:45.401373 1567 catalog_manager.cc:5495] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:45.404897 1860 raft_consensus.cc:348] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.405784 1860 raft_consensus.cc:374] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.406064 1860 raft_consensus.cc:729] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.406739 1860 consensus_queue.cc:260] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.409932 1917 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.413187 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:45.414939 1667 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.415529 1860 ts_tablet_manager.cc:1428] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20260619 21:15:45.416098 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:45.418227 1710 leader_election.cc:341] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.419790 1860 tablet_bootstrap.cc:492] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.414703 1917 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.429744 1667 raft_consensus.cc:2454] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:45.430455 1743 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:45.431768 1634 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.433300 1708 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.433837 1883 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.434352 1883 raft_consensus.cc:686] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:45.434409 1917 raft_consensus.cc:2781] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.434971 1917 raft_consensus.cc:686] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:45.435330 1883 consensus_queue.cc:237] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.436074 1917 consensus_queue.cc:237] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.451268 1860 tablet_bootstrap.cc:654] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.456480 1859 tablet_bootstrap.cc:492] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.456923 1859 ts_tablet_manager.cc:1397] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.098s user 0.014s sys 0.001s
I20260619 21:15:45.459223 1859 raft_consensus.cc:348] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.459761 1859 raft_consensus.cc:374] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.460043 1859 raft_consensus.cc:729] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.459643 1926 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.460656 1859 consensus_queue.cc:260] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.461963 1926 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.461087 1567 catalog_manager.cc:5495] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:45.468134 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.468299 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:45.469744 1817 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:45.470585 1634 leader_election.cc:341] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.449887 1565 catalog_manager.cc:5495] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:45.487074 1817 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:45.488452 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.489784 1883 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.490343 1883 raft_consensus.cc:686] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:45.489468 1566 catalog_manager.cc:5495] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:45.491257 1883 consensus_queue.cc:237] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.495040 1818 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.498296 1859 ts_tablet_manager.cc:1428] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260619 21:15:45.499935 1859 tablet_bootstrap.cc:492] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.503008 1858 tablet_bootstrap.cc:492] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.503631 1858 ts_tablet_manager.cc:1397] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.126s user 0.014s sys 0.017s
I20260619 21:15:45.511118 1859 tablet_bootstrap.cc:654] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.512440 1858 raft_consensus.cc:348] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.513234 1858 raft_consensus.cc:374] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.513556 1858 raft_consensus.cc:729] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.515375 1858 consensus_queue.cc:260] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.516754 1917 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.517287 1917 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.519550 1917 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.521462 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.522133 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.523262 1667 raft_consensus.cc:2379] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:45.524744 1708 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.526364 1917 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.529763 1860 tablet_bootstrap.cc:492] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.530237 1860 ts_tablet_manager.cc:1397] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.111s user 0.022s sys 0.024s
I20260619 21:15:45.532966 1860 raft_consensus.cc:348] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.533690 1860 raft_consensus.cc:374] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.533931 1860 raft_consensus.cc:729] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.529424 1926 consensus_queue.cc:1035] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:45.534610 1860 consensus_queue.cc:260] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.536136 1917 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.536262 1916 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.526451 1858 ts_tablet_manager.cc:1428] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.022s user 0.007s sys 0.001s
I20260619 21:15:45.536710 1916 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.537731 1858 tablet_bootstrap.cc:492] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:45.538885 1916 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.540575 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.540597 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:45.542119 1708 leader_election.cc:341] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:45.542757 1710 leader_election.cc:341] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.543321 1710 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.545261 1858 tablet_bootstrap.cc:654] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.549173 1916 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.555126 1860 ts_tablet_manager.cc:1428] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260619 21:15:45.556324 1860 tablet_bootstrap.cc:492] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.561059 1567 catalog_manager.cc:5495] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:45.562934 1858 tablet_bootstrap.cc:492] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.563161 1859 tablet_bootstrap.cc:492] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.563472 1858 ts_tablet_manager.cc:1397] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.004s
I20260619 21:15:45.563618 1859 ts_tablet_manager.cc:1397] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.064s user 0.019s sys 0.020s
I20260619 21:15:45.563010 1567 catalog_manager.cc:5495] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:45.566790 1858 raft_consensus.cc:348] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.566730 1859 raft_consensus.cc:348] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.567618 1858 raft_consensus.cc:374] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.567764 1859 raft_consensus.cc:374] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.567927 1858 raft_consensus.cc:729] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.568185 1859 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.568717 1858 consensus_queue.cc:260] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.570945 1858 ts_tablet_manager.cc:1428] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260619 21:15:45.572682 1858 tablet_bootstrap.cc:492] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:45.576179 1859 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.578537 1859 ts_tablet_manager.cc:1428] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260619 21:15:45.581642 1858 tablet_bootstrap.cc:654] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.583124 1859 tablet_bootstrap.cc:492] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.587870 1860 tablet_bootstrap.cc:654] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.589989 1859 tablet_bootstrap.cc:654] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.589581 1926 consensus_queue.cc:1035] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:45.618459 1958 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.618937 1958 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.620790 1958 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.621500 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.621981 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.622675 1743 raft_consensus.cc:2379] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:45.623555 1783 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.624191 1958 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.638875 1860 tablet_bootstrap.cc:492] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.638991 1859 tablet_bootstrap.cc:492] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.638962 1858 tablet_bootstrap.cc:492] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.639549 1860 ts_tablet_manager.cc:1397] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.084s user 0.014s sys 0.004s
I20260619 21:15:45.639596 1859 ts_tablet_manager.cc:1397] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.010s
I20260619 21:15:45.639719 1858 ts_tablet_manager.cc:1397] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.067s user 0.008s sys 0.012s
I20260619 21:15:45.642258 1860 raft_consensus.cc:348] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.642583 1859 raft_consensus.cc:348] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.643016 1860 raft_consensus.cc:374] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.643301 1859 raft_consensus.cc:374] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.642814 1858 raft_consensus.cc:348] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.643353 1860 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.643604 1859 raft_consensus.cc:729] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.643980 1858 raft_consensus.cc:374] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.644093 1958 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.644326 1858 raft_consensus.cc:729] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.644411 1859 consensus_queue.cc:260] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.644392 1860 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.645231 1858 consensus_queue.cc:260] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.646693 1860 ts_tablet_manager.cc:1428] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260619 21:15:45.647673 1860 tablet_bootstrap.cc:492] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.650363 1859 ts_tablet_manager.cc:1428] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260619 21:15:45.651422 1859 tablet_bootstrap.cc:492] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.653702 1860 tablet_bootstrap.cc:654] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.657342 1859 tablet_bootstrap.cc:654] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.669224 1858 ts_tablet_manager.cc:1428] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260619 21:15:45.669521 1859 tablet_bootstrap.cc:492] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.669986 1859 ts_tablet_manager.cc:1397] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20260619 21:15:45.670680 1858 tablet_bootstrap.cc:492] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:45.672509 1859 raft_consensus.cc:348] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.673207 1859 raft_consensus.cc:374] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.673465 1859 raft_consensus.cc:729] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.674037 1859 consensus_queue.cc:260] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.676509 1859 ts_tablet_manager.cc:1428] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 21:15:45.680756 1859 tablet_bootstrap.cc:492] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.683720 1860 tablet_bootstrap.cc:492] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.684186 1860 ts_tablet_manager.cc:1397] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.037s user 0.023s sys 0.010s
I20260619 21:15:45.680737 1858 tablet_bootstrap.cc:654] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.689570 1916 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.690058 1916 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.692063 1916 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.693390 1860 raft_consensus.cc:348] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.694152 1860 raft_consensus.cc:374] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.694410 1860 raft_consensus.cc:729] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.694511 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.695091 1860 consensus_queue.cc:260] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:45.696512 1710 leader_election.cc:341] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.697433 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:45.699317 1708 leader_election.cc:341] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.699784 1708 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.700644 1860 ts_tablet_manager.cc:1428] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20260619 21:15:45.701452 1916 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.701862 1858 tablet_bootstrap.cc:492] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.701910 1860 tablet_bootstrap.cc:492] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.702512 1858 ts_tablet_manager.cc:1397] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.007s
I20260619 21:15:45.705667 1858 raft_consensus.cc:348] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.706477 1858 raft_consensus.cc:374] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.706770 1858 raft_consensus.cc:729] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.707449 1858 consensus_queue.cc:260] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.707979 1859 tablet_bootstrap.cc:654] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.709545 1860 tablet_bootstrap.cc:654] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.709699 1858 ts_tablet_manager.cc:1428] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 21:15:45.715412 1858 tablet_bootstrap.cc:492] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:45.721879 1858 tablet_bootstrap.cc:654] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.733930 1860 tablet_bootstrap.cc:492] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.734103 1859 tablet_bootstrap.cc:492] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.734501 1860 ts_tablet_manager.cc:1397] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.012s
I20260619 21:15:45.734638 1859 ts_tablet_manager.cc:1397] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.017s
I20260619 21:15:45.737687 1860 raft_consensus.cc:348] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.737969 1885 consensus_queue.cc:1035] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:45.738481 1860 raft_consensus.cc:374] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.738382 1859 raft_consensus.cc:348] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.739208 1859 raft_consensus.cc:374] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.739499 1859 raft_consensus.cc:729] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.740135 1859 consensus_queue.cc:260] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.740854 1858 tablet_bootstrap.cc:492] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.741339 1858 ts_tablet_manager.cc:1397] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.002s
I20260619 21:15:45.742388 1859 ts_tablet_manager.cc:1428] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260619 21:15:45.738852 1860 raft_consensus.cc:729] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.743687 1858 raft_consensus.cc:348] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.744393 1858 raft_consensus.cc:374] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.744675 1858 raft_consensus.cc:729] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.743801 1860 consensus_queue.cc:260] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.746063 1858 consensus_queue.cc:260] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.746675 1860 ts_tablet_manager.cc:1428] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260619 21:15:45.747651 1860 tablet_bootstrap.cc:492] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.748246 1858 ts_tablet_manager.cc:1428] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260619 21:15:45.756016 1859 tablet_bootstrap.cc:492] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.757197 1858 tablet_bootstrap.cc:492] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:45.759471 1926 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.760033 1926 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.763510 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:45.765167 1634 leader_election.cc:341] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.769183 1958 consensus_queue.cc:1035] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:45.775936 1883 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.776549 1883 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.778981 1883 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.776901 1985 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.780859 1985 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.782465 1958 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.783120 1958 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.785547 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.786464 1741 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:45.787442 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.788620 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:45.789079 1783 leader_election.cc:341] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.789536 1669 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:45.792583 1782 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.793699 1885 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.794133 1885 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.794550 1885 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.805915 1634 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:45.806871 1946 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:45.806939 1883 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.807588 1883 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.807502 1946 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.808012 1883 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.806270 1859 tablet_bootstrap.cc:654] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.808456 1926 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.783540 1985 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.812253 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:45.810837 1946 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.786603 1958 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.812973 1669 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:45.814126 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:45.815405 1783 leader_election.cc:341] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.816519 1782 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.810561 1885 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.817798 1958 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.818449 1958 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.818975 1958 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.820660 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:45.821346 1669 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.821614 1859 tablet_bootstrap.cc:492] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.822063 1859 ts_tablet_manager.cc:1397] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.066s user 0.019s sys 0.035s
I20260619 21:15:45.822845 1883 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.824399 1885 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.824734 1860 tablet_bootstrap.cc:654] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.825176 1859 raft_consensus.cc:348] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.825888 1859 raft_consensus.cc:374] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.826223 1859 raft_consensus.cc:729] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.826789 1859 consensus_queue.cc:260] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.827728 1858 tablet_bootstrap.cc:654] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.833289 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.833904 1742 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.835999 1946 consensus_queue.cc:1035] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:45.829293 1859 ts_tablet_manager.cc:1428] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 21:15:45.839130 1859 tablet_bootstrap.cc:492] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:45.841041 1883 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:45.840921 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.841727 1742 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
W20260619 21:15:45.842507 1783 leader_election.cc:341] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.843545 1634 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:45.844918 1985 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.845409 1985 raft_consensus.cc:686] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:45.846031 1859 tablet_bootstrap.cc:654] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:45.846212 1985 consensus_queue.cc:237] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.852144 1669 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:45.853184 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:45.847164 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:45.860808 1815 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.871292 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.874171 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:45.874501 1635 leader_election.cc:341] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.862280 1782 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.875952 1885 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:45.876443 1885 raft_consensus.cc:686] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:45.858095 1903 consensus_queue.cc:1035] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260619 21:15:45.879050 1885 consensus_queue.cc:237] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.861155 1743 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:45.882057 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:45.883324 1634 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:45.884056 1926 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260619 21:15:45.883313 1635 leader_election.cc:341] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.890693 1858 tablet_bootstrap.cc:492] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:45.890846 1860 tablet_bootstrap.cc:492] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:45.891180 1858 ts_tablet_manager.cc:1397] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.134s user 0.014s sys 0.000s
I20260619 21:15:45.882088 1958 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.891402 1860 ts_tablet_manager.cc:1397] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.144s user 0.009s sys 0.005s
I20260619 21:15:45.887665 1926 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:45.892081 1635 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:45.895824 1926 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:45.900036 1926 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.910250 1860 raft_consensus.cc:348] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.911154 1860 raft_consensus.cc:374] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.911399 1858 raft_consensus.cc:348] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.912143 1858 raft_consensus.cc:374] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.912406 1858 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.911453 1860 raft_consensus.cc:729] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.918218 1958 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:45.914125 1858 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.920847 1858 ts_tablet_manager.cc:1428] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260619 21:15:45.901333 1985 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:45.921940 1858 tablet_bootstrap.cc:492] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:45.920687 1814 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.931542 1815 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
W20260619 21:15:45.931829 1634 consensus_peers.cc:487] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 -> Peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Couldn't send request to peer d53146739e3240239c0a3d5ea90e07fe. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:45.924738 1860 consensus_queue.cc:260] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.933458 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:45.934691 1669 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.935563 1926 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.936686 1859 tablet_bootstrap.cc:492] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:45.937186 1859 ts_tablet_manager.cc:1397] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.098s user 0.028s sys 0.020s
I20260619 21:15:45.937973 1860 ts_tablet_manager.cc:1428] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.046s user 0.009s sys 0.000s
I20260619 21:15:45.939021 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.939659 1741 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:45.927013 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:45.941097 1860 tablet_bootstrap.cc:492] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:45.944811 1926 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:45.942400 1783 leader_election.cc:341] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:45.932525 2015 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260619 21:15:45.946461 1566 catalog_manager.cc:5495] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:45.966558 1859 raft_consensus.cc:348] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.967607 1859 raft_consensus.cc:374] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:45.966060 1958 consensus_queue.cc:1035] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260619 21:15:45.967931 1859 raft_consensus.cc:729] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:45.981498 1916 consensus_queue.cc:1035] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:45.981833 1859 consensus_queue.cc:260] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:45.987000 1669 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:45.980093 1564 catalog_manager.cc:5495] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:46.009528 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:46.018112 1782 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.020117 1946 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.020769 1946 raft_consensus.cc:686] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:46.021844 1946 consensus_queue.cc:237] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.029829 1859 ts_tablet_manager.cc:1428] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.092s user 0.010s sys 0.003s
I20260619 21:15:46.037473 1817 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.049719 2015 consensus_queue.cc:1035] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.057718 1860 tablet_bootstrap.cc:654] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.060467 1916 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.062752 1859 tablet_bootstrap.cc:492] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:46.063563 1885 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.063990 1885 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.065009 1958 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.061645 1916 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.065521 1958 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.066854 1916 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.062480 1741 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:46.069391 1931 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:46.073029 1635 leader_election.cc:341] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.077849 1634 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.069856 1931 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.080291 1931 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.080483 1958 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:46.081667 1885 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:46.081609 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.081952 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.082412 1668 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:46.085832 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.089815 1708 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:46.091205 1917 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:46.091737 1917 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:46.092147 1917 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.091257 1926 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.099020 1926 raft_consensus.cc:686] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:46.100239 1926 consensus_queue.cc:237] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.117677 2015 consensus_queue.cc:1035] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:46.130604 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:46.127408 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 49653us
I20260619 21:15:46.139923 1916 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.140642 1916 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.143301 1916 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.145370 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.146248 1668 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:46.151654 1985 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.152158 1985 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.154193 1859 tablet_bootstrap.cc:654] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.154146 1985 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.163285 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.159245 1916 consensus_queue.cc:1035] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260619 21:15:46.171042 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.175585 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.176258 1858 tablet_bootstrap.cc:654] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.176985 1710 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.177389 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.179289 1916 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.180116 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:46.185882 1634 leader_election.cc:341] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.187651 1916 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.202111 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.203547 1916 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.204056 1916 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.206414 1916 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:46.208454 1634 leader_election.cc:341] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.208837 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.209441 1566 catalog_manager.cc:5495] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:46.205873 1917 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:46.214515 1708 leader_election.cc:341] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.226961 1860 tablet_bootstrap.cc:492] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:46.227463 1860 ts_tablet_manager.cc:1397] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.287s user 0.035s sys 0.047s
I20260619 21:15:46.228144 1917 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:46.224768 1710 leader_election.cc:341] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.231596 1860 raft_consensus.cc:348] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.232357 1860 raft_consensus.cc:374] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.232662 1860 raft_consensus.cc:729] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.235286 1860 consensus_queue.cc:260] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.233430 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:46.239260 1815 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.232427 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.237723 1567 catalog_manager.cc:5495] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:46.244658 1782 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.246155 1860 ts_tablet_manager.cc:1428] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260619 21:15:46.245870 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.247397 2025 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:46.249044 1635 leader_election.cc:341] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.249285 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.249505 1635 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.250195 1926 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.250672 1958 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:46.253266 1635 leader_election.cc:341] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.253738 1635 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.254524 1926 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.257201 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:46.257861 1667 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.263120 1858 tablet_bootstrap.cc:492] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:46.263598 1858 ts_tablet_manager.cc:1397] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.342s user 0.013s sys 0.007s
I20260619 21:15:46.264035 1860 tablet_bootstrap.cc:492] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:46.253762 1815 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
W20260619 21:15:46.268175 1710 leader_election.cc:341] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.268626 1710 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.271214 1667 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:46.272396 1708 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:46.273165 1916 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.273762 1916 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.274168 1916 raft_consensus.cc:686] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:46.274623 1858 raft_consensus.cc:348] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.275349 1858 raft_consensus.cc:374] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.274960 1916 consensus_queue.cc:237] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.275600 1858 raft_consensus.cc:729] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.276207 1858 consensus_queue.cc:260] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.283775 1926 consensus_queue.cc:1035] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:46.286010 1860 tablet_bootstrap.cc:654] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.286175 1858 ts_tablet_manager.cc:1428] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.022s user 0.013s sys 0.000s
I20260619 21:15:46.287111 1858 tablet_bootstrap.cc:492] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:46.297605 1858 tablet_bootstrap.cc:654] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:46.298759 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:46.304499 1859 tablet_bootstrap.cc:492] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:46.304589 1860 tablet_bootstrap.cc:492] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:46.305069 1859 ts_tablet_manager.cc:1397] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.264s user 0.009s sys 0.011s
I20260619 21:15:46.305361 1860 ts_tablet_manager.cc:1397] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.000s
I20260619 21:15:46.305768 1958 consensus_queue.cc:1035] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.308403 1860 raft_consensus.cc:348] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.309305 1860 raft_consensus.cc:374] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.309603 1860 raft_consensus.cc:729] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.310400 1860 consensus_queue.cc:260] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.313290 1860 ts_tablet_manager.cc:1428] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260619 21:15:46.314564 1860 tablet_bootstrap.cc:492] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:46.319252 1567 catalog_manager.cc:5495] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:46.325023 1859 raft_consensus.cc:348] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.325978 1859 raft_consensus.cc:374] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.325829 1931 consensus_queue.cc:1035] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.326398 1859 raft_consensus.cc:729] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.341861 1860 tablet_bootstrap.cc:654] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.343672 1958 consensus_queue.cc:1035] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.340521 1859 consensus_queue.cc:260] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.352248 1926 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.359622 1926 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.369746 1926 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.374704 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.375274 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.377408 1740 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:46.380119 1634 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:15:46.381196 1635 leader_election.cc:341] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.382115 1931 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:46.382395 1859 ts_tablet_manager.cc:1428] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.077s user 0.007s sys 0.002s
I20260619 21:15:46.382633 1931 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:46.383091 1931 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.383822 1859 tablet_bootstrap.cc:492] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:46.414237 1931 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.417146 1931 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.422158 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:46.424201 1635 leader_election.cc:341] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.428254 1931 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.428172 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:46.429569 1742 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.429186 1931 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.433504 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.433686 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:46.442649 1635 leader_election.cc:341] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.443343 1740 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:46.445843 1634 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.448498 1985 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:46.451252 1985 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:46.451882 1985 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.461819 1858 tablet_bootstrap.cc:492] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:46.462447 1858 ts_tablet_manager.cc:1397] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.175s user 0.012s sys 0.007s
I20260619 21:15:46.438805 1931 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:46.472976 1783 consensus_peers.cc:487] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 -> Peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Couldn't send request to peer d53146739e3240239c0a3d5ea90e07fe. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:46.477617 1858 raft_consensus.cc:348] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.479938 1858 raft_consensus.cc:374] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.479647 1946 consensus_queue.cc:1035] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.482537 1858 raft_consensus.cc:729] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.484005 1860 tablet_bootstrap.cc:492] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:46.484459 1860 ts_tablet_manager.cc:1397] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.170s user 0.043s sys 0.037s
I20260619 21:15:46.485628 1858 consensus_queue.cc:260] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.487699 1858 ts_tablet_manager.cc:1428] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260619 21:15:46.488835 1858 tablet_bootstrap.cc:492] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:46.482205 1742 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:46.490698 1634 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.493973 1859 tablet_bootstrap.cc:654] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.510038 1860 raft_consensus.cc:348] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.494097 1926 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.510111 1985 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.511209 1860 raft_consensus.cc:374] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.511839 1860 raft_consensus.cc:729] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.511842 1926 raft_consensus.cc:686] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:46.513160 1926 consensus_queue.cc:237] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.519161 1985 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.524722 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:46.512765 1860 consensus_queue.cc:260] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:46.529608 1635 leader_election.cc:341] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.534420 1860 ts_tablet_manager.cc:1428] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.050s user 0.011s sys 0.002s
I20260619 21:15:46.535778 1860 tablet_bootstrap.cc:492] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:46.538652 1958 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.539388 1958 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.544103 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.546046 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:46.546903 1742 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.554729 1859 tablet_bootstrap.cc:492] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:46.555214 1859 ts_tablet_manager.cc:1397] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.172s user 0.014s sys 0.007s
I20260619 21:15:46.558880 1958 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:46.561028 1931 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.562918 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.563992 1743 raft_consensus.cc:2379] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:46.565500 1783 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.567178 1958 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.577275 1860 tablet_bootstrap.cc:654] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.580564 1858 tablet_bootstrap.cc:654] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.583707 1859 raft_consensus.cc:348] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.588136 1958 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.589190 1859 raft_consensus.cc:374] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.589730 1859 raft_consensus.cc:729] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.590641 1859 consensus_queue.cc:260] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.595932 1859 ts_tablet_manager.cc:1428] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.040s user 0.005s sys 0.003s
I20260619 21:15:46.597406 1859 tablet_bootstrap.cc:492] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:46.597553 1563 catalog_manager.cc:5495] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:46.612581 1858 tablet_bootstrap.cc:492] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:46.613159 1858 ts_tablet_manager.cc:1397] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.125s user 0.020s sys 0.001s
I20260619 21:15:46.615052 1742 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:46.618649 1815 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.618557 1634 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.620481 1883 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.621037 1883 raft_consensus.cc:686] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:46.621923 1883 consensus_queue.cc:237] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.626044 1858 raft_consensus.cc:348] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.626796 1858 raft_consensus.cc:374] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.627096 1858 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.627665 1858 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.627749 2017 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:46.630931 1858 ts_tablet_manager.cc:1428] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260619 21:15:46.632201 1859 tablet_bootstrap.cc:654] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.632668 1858 tablet_bootstrap.cc:492] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:46.641503 1858 tablet_bootstrap.cc:654] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.644215 1860 tablet_bootstrap.cc:492] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:46.644675 1860 ts_tablet_manager.cc:1397] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.109s user 0.007s sys 0.035s
I20260619 21:15:46.657994 1860 raft_consensus.cc:348] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.658859 1860 raft_consensus.cc:374] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.661015 1859 tablet_bootstrap.cc:492] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:46.661621 1859 ts_tablet_manager.cc:1397] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.065s user 0.017s sys 0.011s
I20260619 21:15:46.663367 1916 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.663980 1916 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.665205 1859 raft_consensus.cc:348] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.659160 1860 raft_consensus.cc:729] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.667313 1860 consensus_queue.cc:260] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.666050 1859 raft_consensus.cc:374] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.668977 1859 raft_consensus.cc:729] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.669698 1859 consensus_queue.cc:260] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.672292 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.673768 1883 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.674422 1883 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:46.676144 1710 leader_election.cc:341] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.677457 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.678357 1667 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:46.679427 1883 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.680088 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.672802 1860 ts_tablet_manager.cc:1428] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.028s user 0.008s sys 0.002s
I20260619 21:15:46.681150 1859 ts_tablet_manager.cc:1428] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260619 21:15:46.682160 1860 tablet_bootstrap.cc:492] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:46.682323 1859 tablet_bootstrap.cc:492] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
W20260619 21:15:46.682077 1635 leader_election.cc:341] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.687546 2034 consensus_queue.cc:1035] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260619 21:15:46.686740 1916 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.705960 1859 tablet_bootstrap.cc:654] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.705255 1564 catalog_manager.cc:5495] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:46.690157 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.707903 1742 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:46.689380 1708 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.711078 1916 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:46.711772 1860 tablet_bootstrap.cc:654] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.711570 1916 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:46.712783 1916 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.714318 1634 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.716167 1883 raft_consensus.cc:2781] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:46.716681 1883 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:46.717259 1883 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.725833 2034 consensus_queue.cc:1035] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:46.733718 1883 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.737317 1883 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.738740 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:46.739645 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:46.740597 1742 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:46.742427 1635 leader_election.cc:341] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.743639 1858 tablet_bootstrap.cc:492] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:46.744246 1858 ts_tablet_manager.cc:1397] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.112s user 0.026s sys 0.032s
I20260619 21:15:46.751832 1858 raft_consensus.cc:348] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.752372 1858 raft_consensus.cc:374] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.752626 1858 raft_consensus.cc:729] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.757118 2034 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.757840 2034 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.753237 1858 consensus_queue.cc:260] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.764423 2034 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.767189 1858 ts_tablet_manager.cc:1428] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260619 21:15:46.770910 1858 tablet_bootstrap.cc:492] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:46.763295 1916 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.776039 1916 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.779177 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:46.781322 1860 tablet_bootstrap.cc:492] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:46.782021 1860 ts_tablet_manager.cc:1397] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.100s user 0.017s sys 0.004s
I20260619 21:15:46.785740 1858 tablet_bootstrap.cc:654] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.794981 1860 raft_consensus.cc:348] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.795778 1860 raft_consensus.cc:374] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.796039 1860 raft_consensus.cc:729] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.789783 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:46.797926 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:46.798707 1669 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:46.803093 1710 leader_election.cc:341] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.803882 1860 consensus_queue.cc:260] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.782377 1742 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:46.803772 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.807194 1860 ts_tablet_manager.cc:1428] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.025s user 0.004s sys 0.003s
I20260619 21:15:46.806728 1634 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.813766 1860 tablet_bootstrap.cc:492] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:46.816314 1883 raft_consensus.cc:2781] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.817176 1883 raft_consensus.cc:686] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:46.818442 1883 consensus_queue.cc:237] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.807331 1710 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.827664 1669 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:46.829484 1916 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.836210 1708 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:46.838640 1860 tablet_bootstrap.cc:654] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.838969 1916 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:46.839728 1916 raft_consensus.cc:686] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:46.841869 1859 tablet_bootstrap.cc:492] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:46.841063 1916 consensus_queue.cc:237] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.843179 1859 ts_tablet_manager.cc:1397] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.161s user 0.008s sys 0.022s
I20260619 21:15:46.855818 1741 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.871325 1859 raft_consensus.cc:348] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.872385 1859 raft_consensus.cc:374] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.872709 1859 raft_consensus.cc:729] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.889523 1565 catalog_manager.cc:5495] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:46.900779 1859 consensus_queue.cc:260] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.902595 1860 tablet_bootstrap.cc:492] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:46.903311 1860 ts_tablet_manager.cc:1397] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.017s
I20260619 21:15:46.903770 1859 ts_tablet_manager.cc:1428] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.060s user 0.008s sys 0.000s
I20260619 21:15:46.905126 1859 tablet_bootstrap.cc:492] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:46.907155 1563 catalog_manager.cc:5495] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:46.908092 1860 raft_consensus.cc:348] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.909498 1860 raft_consensus.cc:374] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.909881 1860 raft_consensus.cc:729] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.911341 1860 consensus_queue.cc:260] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.914373 1860 ts_tablet_manager.cc:1428] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.011s user 0.001s sys 0.006s
I20260619 21:15:46.915685 1860 tablet_bootstrap.cc:492] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:46.916057 1858 tablet_bootstrap.cc:492] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:46.916608 1858 ts_tablet_manager.cc:1397] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.146s user 0.027s sys 0.036s
I20260619 21:15:46.922170 2017 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.923082 2017 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.922310 1883 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:46.927904 1883 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.929122 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.927073 2017 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.937014 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.938875 1814 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:46.933719 1883 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:46.935424 1634 leader_election.cc:341] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.940366 1635 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:46.942684 1883 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:46.943104 1883 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:46.937088 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:46.943589 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:46.944394 1742 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
W20260619 21:15:46.945636 1635 leader_election.cc:341] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:46.946231 1635 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:46.950816 2017 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:46.943576 1883 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.952349 1860 tablet_bootstrap.cc:654] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.943634 1858 raft_consensus.cc:348] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.953460 1858 raft_consensus.cc:374] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:46.953750 1858 raft_consensus.cc:729] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:46.955013 1858 consensus_queue.cc:260] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.957669 1858 ts_tablet_manager.cc:1428] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.041s user 0.006s sys 0.004s
I20260619 21:15:46.958922 1858 tablet_bootstrap.cc:492] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:46.960067 1742 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.963452 1859 tablet_bootstrap.cc:654] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.983683 1858 tablet_bootstrap.cc:654] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:46.983587 1883 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:46.987893 1883 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:46.988523 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:46.991947 1743 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:46.991755 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:46.992704 1813 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.011886 1859 tablet_bootstrap.cc:492] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:47.012477 1859 ts_tablet_manager.cc:1397] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.108s user 0.012s sys 0.004s
I20260619 21:15:47.015937 1858 tablet_bootstrap.cc:492] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:47.015801 1859 raft_consensus.cc:348] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.016427 1858 ts_tablet_manager.cc:1397] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.008s
I20260619 21:15:47.016502 1859 raft_consensus.cc:374] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.017181 1859 raft_consensus.cc:729] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.018312 1859 consensus_queue.cc:260] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.024345 1813 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:47.028559 1958 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.029371 1958 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.030588 1858 raft_consensus.cc:348] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.031316 1858 raft_consensus.cc:374] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.031589 1858 raft_consensus.cc:729] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.032280 1858 consensus_queue.cc:260] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.034164 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.034871 1858 ts_tablet_manager.cc:1428] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.018s user 0.011s sys 0.000s
I20260619 21:15:47.036213 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:47.037369 1741 raft_consensus.cc:2379] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:47.038837 1859 ts_tablet_manager.cc:1428] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.026s user 0.007s sys 0.002s
I20260619 21:15:47.039295 1860 tablet_bootstrap.cc:492] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:47.039067 1783 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:47.039724 1860 ts_tablet_manager.cc:1397] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.124s user 0.017s sys 0.025s
I20260619 21:15:47.040392 1885 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.043964 1985 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.045782 1860 raft_consensus.cc:348] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.047262 1860 raft_consensus.cc:374] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.047600 1860 raft_consensus.cc:729] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.048286 1860 consensus_queue.cc:260] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.035956 1958 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:47.057384 1859 tablet_bootstrap.cc:492] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:47.060175 1860 ts_tablet_manager.cc:1428] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.020s user 0.006s sys 0.003s
I20260619 21:15:47.061352 1860 tablet_bootstrap.cc:492] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:47.065070 1858 tablet_bootstrap.cc:492] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:47.057654 1743 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:47.067646 1985 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.069659 1883 consensus_queue.cc:1035] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.066087 1635 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:47.077216 1985 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:47.079085 1885 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:47.077960 1985 raft_consensus.cc:686] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:47.082513 1985 consensus_queue.cc:237] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.095695 1860 tablet_bootstrap.cc:654] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.095824 1817 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.101871 1859 tablet_bootstrap.cc:654] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.109345 1946 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.115659 1946 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.119133 1946 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:47.124061 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:47.129659 1858 tablet_bootstrap.cc:654] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.137143 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.140372 1667 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:47.138470 1741 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:47.143100 1783 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.144678 1885 raft_consensus.cc:2781] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:47.145119 1885 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:47.145491 1885 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.148798 1565 catalog_manager.cc:5495] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:47.178153 1860 tablet_bootstrap.cc:492] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:47.179327 1860 ts_tablet_manager.cc:1397] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.118s user 0.014s sys 0.027s
I20260619 21:15:47.182899 1859 tablet_bootstrap.cc:492] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:47.183871 1859 ts_tablet_manager.cc:1397] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.132s user 0.007s sys 0.018s
I20260619 21:15:47.198617 1946 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.199535 1946 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.201318 1860 raft_consensus.cc:348] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.202200 1860 raft_consensus.cc:374] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.202490 1860 raft_consensus.cc:729] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.202623 2017 consensus_queue.cc:1035] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.203239 1860 consensus_queue.cc:260] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.205477 1860 ts_tablet_manager.cc:1428] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.026s user 0.004s sys 0.007s
I20260619 21:15:47.206684 1859 raft_consensus.cc:348] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.207433 1859 raft_consensus.cc:374] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.207739 1859 raft_consensus.cc:729] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.208630 1859 consensus_queue.cc:260] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.211692 1859 ts_tablet_manager.cc:1428] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20260619 21:15:47.212985 1859 tablet_bootstrap.cc:492] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:47.212741 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.213342 1946 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:47.221405 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:47.222985 1860 tablet_bootstrap.cc:492] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:47.223860 1783 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:47.240955 1858 tablet_bootstrap.cc:492] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:47.241698 1858 ts_tablet_manager.cc:1397] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.177s user 0.029s sys 0.010s
I20260619 21:15:47.235757 1916 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.250828 1858 raft_consensus.cc:348] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.251624 1858 raft_consensus.cc:374] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.251896 1858 raft_consensus.cc:729] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.252502 1858 consensus_queue.cc:260] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.244590 1916 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.259865 1917 consensus_queue.cc:1035] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:47.255402 1885 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.264683 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:47.265712 1669 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.266789 1916 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.267705 1859 tablet_bootstrap.cc:654] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.269327 1858 ts_tablet_manager.cc:1428] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20260619 21:15:47.270746 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.272287 1958 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:47.272897 1668 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:47.270874 1885 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:47.275643 1858 tablet_bootstrap.cc:492] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:47.278483 1860 tablet_bootstrap.cc:654] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.273602 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.281155 1708 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.282117 1815 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:47.282653 1916 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:47.283262 1916 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:47.283803 1916 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.292146 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:47.293071 1743 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:47.295658 1710 consensus_peers.cc:487] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:47.307461 1743 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:47.308676 1917 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.309500 1917 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.309484 1783 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.312186 1946 raft_consensus.cc:2781] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:47.312995 1946 raft_consensus.cc:686] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:47.322149 1946 consensus_queue.cc:237] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.307461 1669 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:47.339628 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.340277 1917 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.340584 1669 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:47.341558 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.341979 1708 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:15:47.343937 1710 leader_election.cc:341] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.345072 1917 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:47.346421 1917 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:47.348256 1917 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.349437 1858 tablet_bootstrap.cc:654] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.368517 2031 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.369647 2031 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.375602 2031 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.375810 1860 tablet_bootstrap.cc:492] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:47.376622 1860 ts_tablet_manager.cc:1397] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.154s user 0.009s sys 0.008s
I20260619 21:15:47.376490 2031 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.377482 2031 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.380627 1860 raft_consensus.cc:348] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.382174 1860 raft_consensus.cc:374] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.376376 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.382494 1860 raft_consensus.cc:729] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.383800 1860 consensus_queue.cc:260] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:47.393466 1710 leader_election.cc:341] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.394487 1859 tablet_bootstrap.cc:492] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:47.395052 1859 ts_tablet_manager.cc:1397] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.182s user 0.019s sys 0.006s
I20260619 21:15:47.397127 1860 ts_tablet_manager.cc:1428] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.020s user 0.008s sys 0.000s
I20260619 21:15:47.398108 1860 tablet_bootstrap.cc:492] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:47.398506 2015 consensus_queue.cc:1035] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.400501 1916 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.401463 1564 catalog_manager.cc:5495] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:47.403337 1859 raft_consensus.cc:348] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.404090 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:47.404714 1815 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.404510 1859 raft_consensus.cc:374] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.407790 1859 raft_consensus.cc:729] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.409379 1859 consensus_queue.cc:260] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.408712 2031 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.406388 1916 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.411090 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.413393 1859 ts_tablet_manager.cc:1428] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.018s user 0.004s sys 0.002s
I20260619 21:15:47.420782 1860 tablet_bootstrap.cc:654] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:47.425634 1710 leader_election.cc:341] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.436723 1859 tablet_bootstrap.cc:492] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:47.447125 1815 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:47.434140 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.450629 1667 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:47.453137 1710 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.455899 1858 tablet_bootstrap.cc:492] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:47.458186 1858 ts_tablet_manager.cc:1397] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.183s user 0.011s sys 0.021s
I20260619 21:15:47.445561 1917 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.460959 1708 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:47.457185 1916 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:47.462654 1916 raft_consensus.cc:686] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:47.462993 2080 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:47.463572 2080 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:47.464068 2080 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.464372 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:47.465317 1669 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.465543 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.468135 2017 consensus_queue.cc:1035] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.470942 1917 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.473474 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:47.474345 1815 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:47.494241 1708 leader_election.cc:341] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.494234 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:47.496713 1668 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.464062 1916 consensus_queue.cc:237] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.506491 1858 raft_consensus.cc:348] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.516832 1858 raft_consensus.cc:374] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.517454 1858 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.518626 1858 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.498019 1708 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:47.520278 2081 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:47.511499 1859 tablet_bootstrap.cc:654] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.526919 1858 ts_tablet_manager.cc:1428] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.051s user 0.006s sys 0.002s
I20260619 21:15:47.534502 1883 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.537086 1858 tablet_bootstrap.cc:492] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:47.544193 1668 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
W20260619 21:15:47.544541 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:47.532698 1815 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:47.562064 1710 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.563066 1917 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:47.563752 1917 raft_consensus.cc:686] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:47.565253 1917 consensus_queue.cc:237] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.571583 1883 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.572310 1883 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.589969 1883 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.615761 1859 tablet_bootstrap.cc:492] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:47.616618 1859 ts_tablet_manager.cc:1397] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.180s user 0.000s sys 0.024s
I20260619 21:15:47.612870 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.595383 1565 catalog_manager.cc:5495] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:47.613883 1860 tablet_bootstrap.cc:492] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:47.622316 1860 ts_tablet_manager.cc:1397] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.224s user 0.025s sys 0.031s
I20260619 21:15:47.578791 2080 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:47.619611 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 48427.3us
I20260619 21:15:47.588848 1669 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:47.640087 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:47.644269 1635 leader_election.cc:341] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.644441 1985 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260619 21:15:47.648563 1634 leader_election.cc:341] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.649237 1634 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:47.648326 1859 raft_consensus.cc:348] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.653440 1859 raft_consensus.cc:374] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.653795 1859 raft_consensus.cc:729] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.654541 1859 consensus_queue.cc:260] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.656677 2017 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:47.663383 1858 tablet_bootstrap.cc:654] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.670531 2080 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.672525 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:47.674501 1710 leader_election.cc:341] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.672549 1883 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.676062 1883 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.679462 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:47.680250 1669 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.681177 1883 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.683975 1565 catalog_manager.cc:5495] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:47.693279 1860 raft_consensus.cc:348] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.696807 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:47.700093 1860 raft_consensus.cc:374] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260619 21:15:47.707031 1634 leader_election.cc:341] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.707122 1860 raft_consensus.cc:729] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.714861 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:47.719519 1635 leader_election.cc:341] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.725950 1669 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:47.708528 1860 consensus_queue.cc:260] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.727730 1708 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:47.720330 1635 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:47.728789 2080 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:47.729522 2080 raft_consensus.cc:686] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:47.730785 2080 consensus_queue.cc:237] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.729542 2015 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:47.735217 1860 ts_tablet_manager.cc:1428] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.113s user 0.009s sys 0.001s
I20260619 21:15:47.740635 1860 tablet_bootstrap.cc:492] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:47.742376 1926 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.743036 1926 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.750573 1926 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.751358 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:47.752209 1743 raft_consensus.cc:2454] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:47.752301 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.754766 1859 ts_tablet_manager.cc:1428] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.136s user 0.013s sys 0.028s
I20260619 21:15:47.756479 1634 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.757323 1859 tablet_bootstrap.cc:492] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:47.761186 2076 consensus_queue.cc:1035] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260619 21:15:47.767235 1926 raft_consensus.cc:2781] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:47.767786 1926 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:47.768282 1926 raft_consensus.cc:3037] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:47.765383 1635 leader_election.cc:341] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.784085 1926 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.785787 1858 tablet_bootstrap.cc:492] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:47.786396 1858 ts_tablet_manager.cc:1397] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.256s user 0.024s sys 0.051s
I20260619 21:15:47.787920 1860 tablet_bootstrap.cc:654] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.795436 1858 raft_consensus.cc:348] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.798720 1926 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.796343 1858 raft_consensus.cc:374] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.801100 1858 raft_consensus.cc:729] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.796504 1563 catalog_manager.cc:5495] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:47.802604 1858 consensus_queue.cc:260] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.805190 1858 ts_tablet_manager.cc:1428] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.018s user 0.010s sys 0.000s
I20260619 21:15:47.807034 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:47.808133 1741 raft_consensus.cc:3037] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.820076 1858 tablet_bootstrap.cc:492] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:47.836565 1859 tablet_bootstrap.cc:654] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.840595 1858 tablet_bootstrap.cc:654] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.846081 1860 tablet_bootstrap.cc:492] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:47.846686 1860 ts_tablet_manager.cc:1397] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.107s user 0.011s sys 0.007s
I20260619 21:15:47.853869 1859 tablet_bootstrap.cc:492] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:47.854490 1859 ts_tablet_manager.cc:1397] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.097s user 0.023s sys 0.025s
I20260619 21:15:47.861346 1858 tablet_bootstrap.cc:492] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:47.861847 1858 ts_tablet_manager.cc:1397] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.042s user 0.024s sys 0.003s
W20260619 21:15:47.861429 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 56344.9us
I20260619 21:15:47.867262 1858 raft_consensus.cc:348] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.867964 1858 raft_consensus.cc:374] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.868182 1858 raft_consensus.cc:729] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.868722 1858 consensus_queue.cc:260] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.871551 1858 ts_tablet_manager.cc:1428] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
W20260619 21:15:47.872171 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 57128.8us
I20260619 21:15:47.874642 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:47.885690 1858 tablet_bootstrap.cc:492] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:47.866173 1860 raft_consensus.cc:348] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.886224 1860 raft_consensus.cc:374] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.886505 1860 raft_consensus.cc:729] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.890583 1741 raft_consensus.cc:2454] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:47.885289 1859 raft_consensus.cc:348] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.894093 1859 raft_consensus.cc:374] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:47.894479 1859 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:47.894719 1858 tablet_bootstrap.cc:654] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.895365 1859 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.897536 1859 ts_tablet_manager.cc:1428] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.043s user 0.012s sys 0.012s
I20260619 21:15:47.898851 1859 tablet_bootstrap.cc:492] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:47.887322 1860 consensus_queue.cc:260] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.905761 1814 raft_consensus.cc:3037] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:47.906945 1860 ts_tablet_manager.cc:1428] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.060s user 0.008s sys 0.000s
I20260619 21:15:47.902882 2080 consensus_queue.cc:1035] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260619 21:15:47.925276 1634 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:47.927459 2015 raft_consensus.cc:2781] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:47.928179 2015 raft_consensus.cc:686] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:47.929569 2015 consensus_queue.cc:237] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.932677 1946 consensus_queue.cc:1035] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:47.936255 1860 tablet_bootstrap.cc:492] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:47.951290 1859 tablet_bootstrap.cc:654] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:47.934753 1814 raft_consensus.cc:2454] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:47.970948 2080 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.972035 2081 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:47.971743 2080 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.972643 2081 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:47.975474 2081 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.976742 2080 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:47.982581 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:47.982822 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.983637 1667 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
W20260619 21:15:47.985702 1710 leader_election.cc:341] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.987628 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:47.992167 1710 leader_election.cc:341] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:47.992255 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:47.993543 1667 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:47.993852 1708 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:47.995417 2081 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:47.995998 2081 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:47.996488 2081 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:48.002745 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:48.004174 2080 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:48.004706 2080 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:48.005308 2080 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:48.026340 1858 tablet_bootstrap.cc:492] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:48.026935 1858 ts_tablet_manager.cc:1397] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.141s user 0.012s sys 0.028s
I20260619 21:15:48.028692 1946 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:48.029702 1946 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.030337 1858 raft_consensus.cc:348] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.031121 1858 raft_consensus.cc:374] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:48.031409 1858 raft_consensus.cc:729] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:48.031879 1859 tablet_bootstrap.cc:492] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:48.032420 1859 ts_tablet_manager.cc:1397] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.134s user 0.022s sys 0.011s
I20260619 21:15:48.032262 1565 catalog_manager.cc:5495] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:48.036166 1860 tablet_bootstrap.cc:654] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:48.043061 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:48.032037 1858 consensus_queue.cc:260] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.052155 1859 raft_consensus.cc:348] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.053606 1859 raft_consensus.cc:374] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:48.053866 1859 raft_consensus.cc:729] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:48.064266 2080 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.068667 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:48.069428 2081 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:48.070641 1710 leader_election.cc:341] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.083932 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:48.084956 1669 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:48.072494 2080 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:48.054461 1859 consensus_queue.cc:260] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.079049 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:48.079658 1917 consensus_queue.cc:1035] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:48.088245 1783 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:48.089589 1885 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:48.079984 1946 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:48.085083 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:48.091678 1860 tablet_bootstrap.cc:492] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:48.092265 1860 ts_tablet_manager.cc:1397] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.161s user 0.036s sys 0.025s
W20260619 21:15:48.092458 1710 leader_election.cc:341] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.085942 2081 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:48.095430 1858 ts_tablet_manager.cc:1428] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.068s user 0.005s sys 0.003s
I20260619 21:15:48.099135 1858 tablet_bootstrap.cc:492] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:48.101666 1859 ts_tablet_manager.cc:1428] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.069s user 0.010s sys 0.001s
I20260619 21:15:48.102873 1859 tablet_bootstrap.cc:492] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:48.107105 2091 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:48.107841 2091 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.114413 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:48.114624 2091 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:48.118655 1634 leader_election.cc:341] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.120479 1860 raft_consensus.cc:348] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.122962 1860 raft_consensus.cc:374] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:48.123325 1860 raft_consensus.cc:729] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:48.124059 1860 consensus_queue.cc:260] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.121764 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:48.126660 1666 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:48.128032 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:48.130609 1916 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:48.133291 1635 leader_election.cc:341] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.139124 1635 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:48.139221 1916 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.136180 2081 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:48.141485 1926 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:48.141642 2081 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.143155 1669 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:48.145083 1666 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:48.147102 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:48.151443 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:48.152591 2080 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
W20260619 21:15:48.153277 1710 leader_election.cc:341] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.153236 1708 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:48.153578 2080 raft_consensus.cc:686] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:48.155072 2080 consensus_queue.cc:237] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.157449 1860 ts_tablet_manager.cc:1428] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.065s user 0.004s sys 0.005s
I20260619 21:15:48.159046 2081 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:48.160727 1917 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:48.161270 1916 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:48.161481 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:48.162565 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:48.170059 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:48.171424 1917 raft_consensus.cc:686] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:48.172322 1860 tablet_bootstrap.cc:492] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
W20260619 21:15:48.172773 1708 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.172729 1917 consensus_queue.cc:237] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:48.174237 1708 leader_election.cc:341] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.174887 1708 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:15:48.175812 1710 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.176409 1710 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:48.177284 2080 raft_consensus.cc:2726] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:48.177966 1916 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:48.249266 1859 tablet_bootstrap.cc:654] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:48.262578 1860 tablet_bootstrap.cc:654] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:48.265655 1858 tablet_bootstrap.cc:654] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:48.260248 2103 consensus_queue.cc:1035] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260619 21:15:48.285866 1814 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:48.283483 2081 consensus_queue.cc:1035] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:48.292528 1565 catalog_manager.cc:5495] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260619 21:15:48.311379 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 53515.5us
I20260619 21:15:48.325501 1815 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:48.326375 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 72928.1us
I20260619 21:15:48.339637 1859 tablet_bootstrap.cc:492] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:48.343393 1859 ts_tablet_manager.cc:1397] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.241s user 0.017s sys 0.022s
I20260619 21:15:48.359143 1565 catalog_manager.cc:5495] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:48.365362 1858 tablet_bootstrap.cc:492] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:48.366120 1858 ts_tablet_manager.cc:1397] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.267s user 0.018s sys 0.025s
I20260619 21:15:48.361516 2109 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260619 21:15:48.405709 1859 raft_consensus.cc:348] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.406585 1859 raft_consensus.cc:374] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:48.406881 1859 raft_consensus.cc:729] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:48.407580 1859 consensus_queue.cc:260] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.415894 1860 tablet_bootstrap.cc:492] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:48.416448 1860 ts_tablet_manager.cc:1397] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.244s user 0.011s sys 0.020s
I20260619 21:15:48.411021 1858 raft_consensus.cc:348] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.420008 1858 raft_consensus.cc:374] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:48.419826 1860 raft_consensus.cc:348] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.420559 1859 ts_tablet_manager.cc:1428] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.070s user 0.007s sys 0.000s
I20260619 21:15:48.421025 1858 raft_consensus.cc:729] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:48.422093 1859 tablet_bootstrap.cc:492] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:48.422935 1858 consensus_queue.cc:260] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.421178 1860 raft_consensus.cc:374] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:48.425539 1860 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:48.437444 2108 consensus_queue.cc:1035] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260619 21:15:48.431962 2109 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260619 21:15:48.451524 2091 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:48.452234 2091 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.454747 2091 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:48.403079 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 46073.7us
I20260619 21:15:48.498857 1860 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:48.506538 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:48.511566 1858 ts_tablet_manager.cc:1428] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.145s user 0.027s sys 0.017s
I20260619 21:15:48.512967 1858 tablet_bootstrap.cc:492] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
W20260619 21:15:48.517761 1635 leader_election.cc:341] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:48.522945 1860 ts_tablet_manager.cc:1428] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.106s user 0.032s sys 0.040s
I20260619 21:15:48.551911 1859 tablet_bootstrap.cc:654] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:48.573297 1860 tablet_bootstrap.cc:492] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:48.573526 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:48.584236 1740 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:48.609484 2108 consensus_queue.cc:1035] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260619 21:15:48.633550 1812 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:48.703303 1858 tablet_bootstrap.cc:654] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:48.726934 1860 tablet_bootstrap.cc:654] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:48.746025 1710 consensus_peers.cc:487] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:48.720906 1634 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
W20260619 21:15:48.847069 2118 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.098s user 0.000s sys 0.001s
I20260619 21:15:48.856493 1859 tablet_bootstrap.cc:492] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:48.857183 1859 ts_tablet_manager.cc:1397] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.435s user 0.021s sys 0.048s
I20260619 21:15:48.958127 1917 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:49.093245 2118 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.106s user 0.000s sys 0.000s
I20260619 21:15:48.995352 1917 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:49.416354 2114 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:49.417224 2114 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:49.539186 1784 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.523s user 0.016s sys 0.005s
W20260619 21:15:50.053800 1708 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.873s user 0.023s sys 0.043s
W20260619 21:15:50.054342 1708 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.873s user 0.023s sys 0.043s
I20260619 21:15:49.560500 1917 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.079288 1917 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.080125 1917 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.083029 1917 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.084420 1917 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:15:50.085772 1815 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.540s user 0.085s sys 0.202s
W20260619 21:15:50.088917 1815 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.543s user 0.085s sys 0.202s
I20260619 21:15:50.086606 1917 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.099982 2130 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.100148 2128 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.100971 2129 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.100672 2130 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.101662 2125 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.101549 2129 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.102252 2125 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.103147 2130 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.103678 2130 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.103775 2129 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.104228 2125 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.104300 2129 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.100757 2128 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.104157 2130 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.112748 1858 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.537s user 0.002s sys 0.001s
W20260619 21:15:50.113164 1858 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.538s user 0.002s sys 0.001s
W20260619 21:15:50.115373 1710 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.774s user 0.000s sys 0.004s
W20260619 21:15:50.115832 1710 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.779s user 0.000s sys 0.004s
I20260619 21:15:50.119526 2135 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.120431 2135 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.128003 1860 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.572s user 0.002s sys 0.006s
W20260619 21:15:50.128463 1860 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.573s user 0.002s sys 0.006s
I20260619 21:15:50.100029 2074 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.132467 1858 tablet_bootstrap.cc:492] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:50.133131 1858 ts_tablet_manager.cc:1397] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 1.621s user 0.022s sys 0.001s
I20260619 21:15:50.104816 2129 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.131875 1707 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.747s user 0.023s sys 0.048s
W20260619 21:15:50.132256 1781 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.569s user 0.007s sys 0.002s
I20260619 21:15:50.133378 2126 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.134647 2124 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.134778 2074 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.139654 1781 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.576s user 0.007s sys 0.003s
W20260619 21:15:50.141144 1707 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.832s user 0.024s sys 0.048s
I20260619 21:15:50.140067 2125 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.153962 2130 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.154341 2074 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.155323 2129 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.155921 1860 tablet_bootstrap.cc:492] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:50.161729 1860 ts_tablet_manager.cc:1397] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.589s user 0.018s sys 0.026s
I20260619 21:15:50.156409 2133 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.147650 2128 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:50.146533 1709 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.761s user 0.018s sys 0.020s
I20260619 21:15:50.157935 1917 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.158499 2125 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.164501 2138 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.157171 2124 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.166168 2125 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.166276 2124 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.162667 2074 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.162724 1858 raft_consensus.cc:348] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.168000 1858 raft_consensus.cc:374] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:50.165792 2133 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.161334 2137 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:15:50.166925 1709 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.819s user 0.019s sys 0.020s
I20260619 21:15:50.166973 2129 consensus_queue.cc:1035] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260619 21:15:50.168246 1858 raft_consensus.cc:729] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:50.175969 1860 raft_consensus.cc:348] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.176563 1860 raft_consensus.cc:374] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:50.176798 1860 raft_consensus.cc:729] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:50.170359 2074 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.179898 2074 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.157408 2126 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.175758 1858 consensus_queue.cc:260] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.181456 1860 consensus_queue.cc:260] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.161631 2130 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.187928 2141 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.185081 2130 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.186596 2128 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.189467 2129 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.191174 2124 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.191318 2132 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.193908 1860 ts_tablet_manager.cc:1428] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260619 21:15:50.191411 2129 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.191769 2124 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.192960 2128 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.196810 1858 ts_tablet_manager.cc:1428] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.063s user 0.005s sys 0.002s
I20260619 21:15:50.188611 2141 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.191825 2132 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.200302 1860 tablet_bootstrap.cc:492] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:50.196308 2126 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.202778 2139 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:50.194725 2125 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.206105 1858 tablet_bootstrap.cc:492] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:50.207561 2136 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.201155 2074 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.173902 2137 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.214238 2128 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.221438 2074 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.174470 2138 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.209380 2136 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.207082 2125 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.223027 2137 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.225364 2124 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:15:50.244043 2136 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.241654 2132 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.244913 2130 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.245867 2142 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.203485 2139 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.221989 2133 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.246752 2129 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.241138 2138 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.248350 2141 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.246454 2142 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.253212 2139 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.254252 2142 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.327039 2142 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.327764 2127 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.327922 2142 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.328630 2127 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.331051 2142 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.332289 2127 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.344183 2074 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.352021 2135 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.395686 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.398067 2127 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.398218 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.399252 1818 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.399151 2127 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.402123 2127 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.437815 1858 tablet_bootstrap.cc:654] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:50.442000 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.444356 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.445122 1809 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.445730 2127 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.445889 2074 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.446445 2127 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.447065 1858 tablet_bootstrap.cc:492] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:50.447835 1858 ts_tablet_manager.cc:1397] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.246s user 0.012s sys 0.003s
I20260619 21:15:50.449721 2127 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.452416 2114 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.453615 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.446445 2074 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.454538 1809 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.457734 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.459653 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.460568 1818 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.461983 2074 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.462743 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.463040 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.463474 1818 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.464447 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.464887 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.465236 1816 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.466786 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.467449 1809 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.476794 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.481237 2125 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.483237 1917 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.483565 2132 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.484107 1917 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.484406 2132 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.486379 1917 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.486490 2132 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.487824 2135 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.488524 2135 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.489039 1710 leader_election.cc:341] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:50.490113 1710 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.491496 2128 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.492002 2128 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.491843 1710 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.492607 2128 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.493789 1710 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.495477 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.496372 1817 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.496395 1710 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.497185 2139 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.497783 2141 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.498375 2141 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.498911 2141 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.493597 2143 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.500375 2143 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.500897 2143 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.497747 2139 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.492409 2135 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.502136 2137 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.502002 2139 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.503216 2138 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.511798 2129 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.505024 1917 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.512321 2129 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.512473 1917 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.511358 2138 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.505723 2137 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.529727 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 1.81454e+06us
W20260619 21:15:50.530695 1710 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:50.539851 2017 raft_consensus.cc:2781] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:50.540400 1710 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.541036 2015 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.540489 2017 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:50.541746 2017 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:50.541667 2015 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.503392 2132 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.569199 1858 raft_consensus.cc:348] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.570401 1565 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37734 (request call id 72) took 1780 ms. Trace:
I20260619 21:15:50.571012 1858 raft_consensus.cc:374] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:50.571475 1858 raft_consensus.cc:729] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:50.574056 1859 raft_consensus.cc:348] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.574779 1859 raft_consensus.cc:374] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:50.574290 1858 consensus_queue.cc:260] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.575054 1859 raft_consensus.cc:729] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:50.576473 2135 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.577268 1859 consensus_queue.cc:260] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.577895 2135 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.578079 2015 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.578275 2135 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.579245 1710 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.584028 1858 ts_tablet_manager.cc:1428] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.136s user 0.008s sys 0.011s
W20260619 21:15:50.584959 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 45394.8us
I20260619 21:15:50.576918 2132 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.611836 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.575359 1565 rpcz_store.cc:276] 0619 21:15:48.773283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:48.773466 (+ 183us) service_pool.cc:227] Handling call
0619 21:15:50.553401 (+1779935us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":3}
I20260619 21:15:50.616505 2138 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.620674 1859 ts_tablet_manager.cc:1428] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 1.763s user 0.023s sys 0.021s
I20260619 21:15:50.595806 2129 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.633518 1860 tablet_bootstrap.cc:654] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:50.634606 1858 tablet_bootstrap.cc:492] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
W20260619 21:15:50.621071 1859 ts_tablet_manager.cc:1460] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Tablet startup took 1763ms
W20260619 21:15:50.640596 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:50.647953 2132 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.651302 2143 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:50.636833 1859 ts_tablet_manager.cc:1462] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Trace:
0619 21:15:42.251532 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:42.728343 (+476811us) service_pool.cc:227] Handling call
0619 21:15:42.730175 (+ 1832us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:15:42.730264 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:15:42.848078 (+117814us) tablet_metadata.cc:641] Metadata flushed
0619 21:15:42.942128 (+ 94050us) tablet_replica.cc:205] Creating consensus instance
0619 21:15:42.942637 (+ 509us) inbound_call.cc:173] Queueing success response
0619 21:15:48.421741 (+5479104us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:15:48.857739 (+435998us) ts_tablet_manager.cc:1429] Starting tablet replica
0619 21:15:49.548520 (+690781us) tablet_replica.cc:259] Starting instrumentation
0619 21:15:49.552353 (+ 3833us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":960629,"raft.queue_time_us":20207,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":2810,"tablet-open.queue_time_us":5479185}
I20260619 21:15:50.638005 2129 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.664297 1859 tablet_bootstrap.cc:492] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:50.645515 1917 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.640027 2137 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.663626 2129 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.681550 1883 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.686582 1883 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.663319 2015 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:15:50.665834 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 80181.5us
I20260619 21:15:50.665796 2091 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:15:50.683713 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 104566us
I20260619 21:15:50.695760 1917 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.696534 1917 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.692142 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.700919 2150 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:50.689173 2108 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.680413 2137 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.688402 2129 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.706867 1883 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.707813 1883 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.692341 2132 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.694453 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.711925 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.714274 1883 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.724924 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.710685 2137 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.691000 2015 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.701951 2108 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.698495 2091 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.735574 1883 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.716048 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.716368 2143 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.733819 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.739619 1743 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:50.739836 2147 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:50.730659 1741 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:50.701807 2150 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.727885 2132 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.750612 2125 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:50.731194 1710 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.750840 1917 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.753221 2125 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.744508 2141 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.740875 2147 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.749046 2128 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.756189 2135 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.758997 2137 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.769913 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.761184 2138 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.766714 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.774170 1739 raft_consensus.cc:2454] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:50.778918 1740 raft_consensus.cc:2379] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:50.785562 1858 tablet_bootstrap.cc:654] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:50.775300 2138 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.788792 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.787849 1738 raft_consensus.cc:2379] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:50.791819 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.782755 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.793442 1741 raft_consensus.cc:2379] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:50.793296 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.793525 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.794239 1735 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:50.794374 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.796248 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.796864 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.797295 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.798202 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.798552 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.799023 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.799482 1735 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:50.799741 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.800280 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.779635 2135 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.802595 2015 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.803313 2150 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.804214 2132 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.804661 2091 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.810494 1710 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.811486 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.812645 2132 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.812718 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.813128 2132 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.813788 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.815232 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.786823 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.791122 2138 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.808712 1736 raft_consensus.cc:2379] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:50.821158 2108 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.824546 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.817698 2132 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:50.816346 2128 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.829277 1740 raft_consensus.cc:1226] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:50.828146 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.815379 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.822608 2147 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.825220 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.847364 1743 raft_consensus.cc:1226] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:50.850715 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.847585 1783 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:50.848424 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.849671 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.836520 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.856740 1860 tablet_bootstrap.cc:492] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:50.857251 1860 ts_tablet_manager.cc:1397] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.660s user 0.017s sys 0.027s
I20260619 21:15:50.860682 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.839910 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.861704 1810 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:50.858927 2141 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.866142 2114 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:50.867964 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.869516 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.877905 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.871497 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.871905 2139 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.873414 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.875720 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:50.875851 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.875833 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.869838 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.879626 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.881134 1740 raft_consensus.cc:1226] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:50.882369 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.883067 1859 tablet_bootstrap.cc:654] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:50.883643 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.868791 2114 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:50.865273 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.892084 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.864228 2125 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:50.894910 2129 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.905282 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.905730 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.899617 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.907683 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.899617 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.909207 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.912390 2091 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.003474 2091 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.980384 1883 consensus_queue.cc:1046] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260619 21:15:50.974815 1860 raft_consensus.cc:348] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:50.995165 2139 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:50.987883 2155 consensus_queue.cc:1046] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260619 21:15:51.007026 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.007690 2108 raft_consensus.cc:3032] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.007611 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.008402 2108 raft_consensus.cc:729] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:50.985850 1744 raft_consensus.cc:2379] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:51.010039 2138 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.011796 1736 raft_consensus.cc:1226] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.012243 2129 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.013221 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.009963 2108 consensus_queue.cc:260] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.006287 1860 raft_consensus.cc:374] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:51.009940 1737 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:51.015404 1860 raft_consensus.cc:729] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:51.016425 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.969661 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.017791 1742 raft_consensus.cc:1226] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.018183 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.018741 2108 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.009196 1666 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:51.025000 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.025677 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.031076 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.024710 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.032642 1782 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.033188 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.034996 1782 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.020017 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.016383 1860 consensus_queue.cc:260] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.027387 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.006306 2147 raft_consensus.cc:3032] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.055764 1858 tablet_bootstrap.cc:492] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:51.049551 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.014210 2114 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.051000 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.052037 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.053280 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:50.903416 1783 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:51.055605 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.056279 1858 ts_tablet_manager.cc:1397] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.422s user 0.016s sys 0.024s
I20260619 21:15:51.055847 2147 raft_consensus.cc:729] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.062806 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.062707 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.064299 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.000070 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:50.980706 1634 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:50.991250 2017 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.070434 1946 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.099143 2137 consensus_queue.cc:1035] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:51.070541 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.004488 2132 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.072480 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.073721 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.036077 1662 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:51.020768 2091 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.075345 1817 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:51.075275 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.076516 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.077160 2129 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.112267 1739 raft_consensus.cc:1226] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.077939 2138 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.057073 2125 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.079219 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.081023 1860 ts_tablet_manager.cc:1428] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.223s user 0.011s sys 0.002s
I20260619 21:15:51.082082 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.082257 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.052441 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.062738 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.087565 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.052073 1735 raft_consensus.cc:2379] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:51.120254 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.121459 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.121973 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.122722 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.123839 1818 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:15:51.124500 1708 leader_election.cc:341] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.037067 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:51.087522 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 197745us
I20260619 21:15:51.125548 1708 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.126209 1783 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:51.126888 2138 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:51.091284 1885 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:51.106885 2149 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.107203 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.107339 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.100863 1917 consensus_queue.cc:1035] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260619 21:15:51.108220 1710 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:15:51.110138 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 75314.3us
I20260619 21:15:51.110081 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.092417 2128 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.152714 2128 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.155560 2015 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.113665 2132 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.112314 2129 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.119678 1860 tablet_bootstrap.cc:492] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:51.101132 2017 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.129736 1708 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.094903 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.128125 1783 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:51.131683 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.133394 2147 consensus_queue.cc:260] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.163826 1858 raft_consensus.cc:348] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.164709 1858 raft_consensus.cc:374] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:51.164999 1858 raft_consensus.cc:729] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:51.167366 1708 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.169770 2074 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:51.170291 2139 raft_consensus.cc:2726] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.168568 2138 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.170322 2074 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:51.172159 2074 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.174214 1783 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:51.175845 2129 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.156553 2128 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.165726 1858 consensus_queue.cc:260] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:51.181721 1708 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.130334 1885 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:15:51.130183 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 150876us
I20260619 21:15:51.147905 1917 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:51.190197 1917 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:51.190821 1917 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.191390 1634 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.193310 1883 consensus_queue.cc:1046] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:51.194146 2150 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.194396 1883 raft_consensus.cc:3032] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.130483 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.194646 2125 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:51.195195 2125 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:51.181874 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.195783 2125 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.184547 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.187429 1708 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.187234 1710 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.189523 1858 ts_tablet_manager.cc:1428] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.101s user 0.009s sys 0.000s
I20260619 21:15:51.189457 1885 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.156391 2015 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.194794 1883 raft_consensus.cc:729] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.138464 1859 tablet_bootstrap.cc:492] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:51.250547 1858 tablet_bootstrap.cc:492] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:51.213970 2132 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.250396 1883 consensus_queue.cc:260] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:51.244269 1690 debug-util.cc:398] Leaking SignalData structure 0x7b08001d0320 after lost signal to thread 399
I20260619 21:15:51.214341 1708 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:15:51.246301 1635 leader_election.cc:341] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.255781 1883 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.176750 1818 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:51.201917 2137 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:51.254405 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.237262 1634 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.256603 1859 ts_tablet_manager.cc:1397] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.593s user 0.061s sys 0.077s
W20260619 21:15:51.264312 1690 debug-util.cc:398] Leaking SignalData structure 0x7b08001d03c0 after lost signal to thread 1601
I20260619 21:15:51.265079 1810 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:51.265522 2108 raft_consensus.cc:2726] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:51.265678 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 47568.6us
I20260619 21:15:51.266014 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:51.266304 1690 debug-util.cc:398] Leaking SignalData structure 0x7b0800086800 after lost signal to thread 1693
I20260619 21:15:51.266749 2108 consensus_queue.cc:1046] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:51.267108 2149 consensus_queue.cc:1046] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:51.267891 2155 raft_consensus.cc:3032] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.153064 2162 raft_consensus.cc:2781] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:51.268163 2149 raft_consensus.cc:3032] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.268224 2155 raft_consensus.cc:729] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.268499 2162 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:51.268633 2149 raft_consensus.cc:729] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.257885 2128 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.269491 1860 tablet_bootstrap.cc:654] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:51.269915 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 70430.9us
I20260619 21:15:51.270813 1708 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.272425 1708 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.261610 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.261319 1917 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.200557 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.249727 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.273384 2149 consensus_queue.cc:260] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.218012 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.292413 2149 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:15:51.297479 1635 leader_election.cc:341] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.268929 2162 raft_consensus.cc:3037] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.288472 1708 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.252828 2015 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.283941 2138 raft_consensus.cc:2726] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.259661 2147 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.273270 2155 consensus_queue.cc:260] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.290162 2132 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:51.311944 1690 debug-util.cc:398] Leaking SignalData structure 0x7b080025f240 after lost signal to thread 1768
I20260619 21:15:51.294785 2137 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.313130 2128 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:51.306568 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 132523us
W20260619 21:15:51.338980 401 debug-util.cc:398] Leaking SignalData structure 0x7b080000e2c0 after lost signal to thread 1634
W20260619 21:15:51.350023 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 61631.2us
I20260619 21:15:51.339318 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.355172 1708 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.351800 2155 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.356372 1708 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.357802 1708 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.342294 2137 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.359618 1708 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.358850 1859 raft_consensus.cc:348] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.360589 1859 raft_consensus.cc:374] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:51.360848 1859 raft_consensus.cc:729] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:51.357270 2132 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.361832 1859 consensus_queue.cc:260] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.359889 2138 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.365197 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.365780 1811 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.366020 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.362772 2139 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.371296 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.372499 1917 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.372745 1710 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.373881 1710 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.378815 1710 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.380296 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.380751 1742 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.372394 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.384413 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.385215 2132 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.387022 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:51.387850 1741 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:51.388499 1735 raft_consensus.cc:1226] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.389065 1813 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.391839 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.393090 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.392238 1811 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:51.381793 2138 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.396098 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.359488 2128 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.397589 1710 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:51.398705 1710 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.399680 1710 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
W20260619 21:15:51.374015 1690 debug-util.cc:398] Leaking SignalData structure 0x7b08001bcba0 after lost signal to thread 1842
I20260619 21:15:51.400492 2141 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.405092 2153 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.410825 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.403594 2135 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:51.405848 2132 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:51.411839 2135 raft_consensus.cc:686] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:51.411890 2132 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:51.382262 1917 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.413146 2135 consensus_queue.cc:237] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.414547 1883 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.415077 1883 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.415858 2108 consensus_queue.cc:1046] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:51.417258 1883 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.381268 1736 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:51.419627 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.420825 2015 raft_consensus.cc:3032] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.421331 2015 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.421844 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.422392 2137 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.423264 1782 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:51.421414 1635 leader_election.cc:341] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.410370 1634 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.424501 1946 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.411453 2153 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.425786 1782 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.427701 2153 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.430742 2108 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:51.431711 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 48634.8us
I20260619 21:15:51.412580 2132 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.422715 2015 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.442832 1859 ts_tablet_manager.cc:1428] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.173s user 0.007s sys 0.000s
W20260619 21:15:51.446905 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 45349.4us
W20260619 21:15:51.444629 1708 leader_election.cc:341] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.452482 1858 tablet_bootstrap.cc:654] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:51.452100 1710 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.454469 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.455972 1669 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:51.455046 1859 tablet_bootstrap.cc:492] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:51.445855 1815 raft_consensus.cc:1226] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.460927 1883 consensus_queue.cc:1046] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:51.395236 1885 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.450208 2164 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.468039 2108 raft_consensus.cc:3032] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.474063 2108 raft_consensus.cc:729] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.460449 2015 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.461783 2129 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.491060 1814 raft_consensus.cc:1226] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.478263 2108 consensus_queue.cc:260] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.482424 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.474148 2114 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.510969 1816 raft_consensus.cc:1226] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.512490 1885 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:51.481205 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.523186 2091 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.525454 1860 tablet_bootstrap.cc:492] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:51.525998 1860 ts_tablet_manager.cc:1397] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.407s user 0.021s sys 0.017s
W20260619 21:15:51.530212 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 57242.9us
I20260619 21:15:51.531577 1634 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.533217 2017 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.521940 2074 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.505517 1946 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:51.556310 1635 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.529122 2108 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.547492 1810 raft_consensus.cc:1226] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.514292 1782 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.568601 2017 consensus_queue.cc:1046] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260619 21:15:51.570546 1741 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
W20260619 21:15:51.573390 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 112603us
W20260619 21:15:51.578123 1710 leader_election.cc:341] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.572146 2017 raft_consensus.cc:3032] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.581163 1710 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.549103 1663 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
W20260619 21:15:51.587409 1710 leader_election.cc:341] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.590090 1710 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:51.591038 1710 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.586267 2017 raft_consensus.cc:729] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.592453 2017 consensus_queue.cc:260] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.594239 2017 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.549238 2150 consensus_queue.cc:1046] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
W20260619 21:15:51.598248 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 137145us
I20260619 21:15:51.598443 2141 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:51.545545 1634 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.589134 2135 raft_consensus.cc:2726] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:51.618005 1860 raft_consensus.cc:348] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.626300 1860 raft_consensus.cc:374] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260619 21:15:51.619690 1708 leader_election.cc:341] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.621016 2141 raft_consensus.cc:686] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:51.621481 2173 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.621580 2074 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:51.598503 2128 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.650058 1860 raft_consensus.cc:729] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:51.650507 1708 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:15:51.616165 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 120755us
I20260619 21:15:51.652133 2141 consensus_queue.cc:237] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:51.728336 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 108663us
W20260619 21:15:51.727922 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 73894.2us
I20260619 21:15:51.763281 2147 raft_consensus.cc:3032] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.738193 2135 raft_consensus.cc:2726] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:51.780530 1708 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:51.781545 2147 raft_consensus.cc:729] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.783003 2164 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.737762 1859 tablet_bootstrap.cc:654] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:51.689116 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:51.712502 1860 consensus_queue.cc:260] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.785218 2176 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:51.741479 1782 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.791128 2176 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:51.626423 2114 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:51.631553 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:51.709637 2155 consensus_queue.cc:1046] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260619 21:15:51.625963 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.794317 1858 tablet_bootstrap.cc:492] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:51.665946 2149 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.792886 1708 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.800855 1858 ts_tablet_manager.cc:1397] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.550s user 0.030s sys 0.050s
I20260619 21:15:51.796970 2176 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:51.796064 1860 ts_tablet_manager.cc:1428] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.270s user 0.011s sys 0.008s
W20260619 21:15:51.723743 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 190477us
I20260619 21:15:51.637286 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:51.631810 2132 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.800040 2164 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.634038 2125 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.850231 1809 raft_consensus.cc:1226] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.765687 2162 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.854663 1816 raft_consensus.cc:1226] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.841315 1634 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.813225 1810 raft_consensus.cc:2379] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
W20260619 21:15:51.832578 1710 leader_election.cc:341] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.834092 1858 raft_consensus.cc:348] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.711360 2153 consensus_queue.cc:1046] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260619 21:15:51.864794 2149 raft_consensus.cc:3032] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.865545 2149 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
W20260619 21:15:51.632444 1783 leader_election.cc:341] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:51.867061 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 234660us
I20260619 21:15:51.866653 2149 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.868319 2149 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:15:51.870713 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 51688.7us
I20260619 21:15:51.872577 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.874424 1710 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.806030 2147 consensus_queue.cc:260] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.875638 2147 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.877460 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.878017 2173 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.877095 2125 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:15:51.879731 2015 consensus_queue.cc:1159] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d53146739e3240239c0a3d5ea90e07fe" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
W20260619 21:15:51.824306 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 82985.5us
I20260619 21:15:51.881572 1782 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.883549 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:51.884889 2176 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.886353 2074 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.887938 2176 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.884305 1782 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.890805 1782 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.892089 2164 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.892925 2164 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.895164 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:51.828613 2135 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:51.841710 1666 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:51.848894 2108 raft_consensus.cc:3032] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.896660 2108 raft_consensus.cc:729] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.898028 2108 consensus_queue.cc:260] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.899883 2108 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.858903 1858 raft_consensus.cc:374] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:51.860548 2155 consensus_queue.cc:1046] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:51.901295 1858 raft_consensus.cc:729] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:51.839093 2132 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.861768 2150 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.902137 1858 consensus_queue.cc:260] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.862036 1634 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.863862 2162 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:51.904706 1858 ts_tablet_manager.cc:1428] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.103s user 0.004s sys 0.001s
I20260619 21:15:51.904716 1634 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.912655 1858 tablet_bootstrap.cc:492] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:51.912951 1634 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.912854 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:51.914088 2153 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:51.914561 2153 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:51.915009 2153 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:51.835371 1860 tablet_bootstrap.cc:492] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:51.907145 2155 raft_consensus.cc:3032] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:51.905732 2015 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:51.920729 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 46370.7us
I20260619 21:15:51.895889 1782 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.924660 1917 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.916148 2155 raft_consensus.cc:729] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.928907 2155 consensus_queue.cc:260] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.931139 2155 raft_consensus.cc:3037] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:51.924873 2017 raft_consensus.cc:2781] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:51.931944 2017 raft_consensus.cc:686] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:51.933346 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:51.933291 2017 consensus_queue.cc:237] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.934523 1739 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:51.926259 2135 raft_consensus.cc:686] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:51.936964 2135 consensus_queue.cc:237] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.937700 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:51.928279 1782 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.942740 1708 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.941737 2173 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.953326 1708 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
W20260619 21:15:51.948659 1710 leader_election.cc:341] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:51.963140 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.964025 1741 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:51.966764 1634 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:51.952834 2139 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.968448 1736 raft_consensus.cc:1226] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.959192 2141 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:51.969760 2015 raft_consensus.cc:2781] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:51.969461 2141 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:51.970237 2015 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:51.970739 2015 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:51.961614 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:51.958225 2162 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.972268 2141 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:51.961732 2175 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:51.974753 2175 raft_consensus.cc:686] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:51.977243 1859 tablet_bootstrap.cc:492] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:51.977809 1859 ts_tablet_manager.cc:1397] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.527s user 0.034s sys 0.043s
I20260619 21:15:51.981142 1738 raft_consensus.cc:1226] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:51.985509 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:51.986178 2074 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:51.986397 1666 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:51.991333 1708 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:51.978812 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:51.995163 2129 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:51.995725 2129 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:51.996307 2129 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:51.998880 2153 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.000505 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:52.001454 1817 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:52.001931 1660 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:52.003441 1809 raft_consensus.cc:1261] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.006901 2132 consensus_queue.cc:1035] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:51.992751 1782 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.008040 1858 tablet_bootstrap.cc:654] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:52.009922 1634 leader_election.cc:341] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.010466 1634 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:51.977072 2175 consensus_queue.cc:237] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.012053 2108 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:52.013922 1660 raft_consensus.cc:1261] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.017665 2162 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:52.019456 2153 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.021001 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:52.022698 1635 leader_election.cc:341] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.016312 1859 raft_consensus.cc:348] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.023622 1859 raft_consensus.cc:374] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.023991 1859 raft_consensus.cc:729] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.013887 1782 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.026923 1708 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:52.026512 1859 consensus_queue.cc:260] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.030565 2139 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:52.034263 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.035431 1739 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:52.040066 1634 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.040653 2074 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:52.032691 2139 consensus_queue.cc:1035] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260619 21:15:52.045428 1859 ts_tablet_manager.cc:1428] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.067s user 0.005s sys 0.003s
I20260619 21:15:51.994643 2017 consensus_queue.cc:1046] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260619 21:15:52.046438 1859 tablet_bootstrap.cc:492] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:52.048878 2108 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:52.056450 2017 raft_consensus.cc:3032] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:52.069270 1782 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.070582 2162 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:52.071040 2162 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:52.071439 2162 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.073915 1782 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.075227 2173 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:52.084764 2162 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.087802 2162 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.059041 2017 raft_consensus.cc:729] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:52.090859 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.093482 2017 consensus_queue.cc:260] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.095562 2017 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:52.096979 1860 tablet_bootstrap.cc:654] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:52.088330 2015 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.102025 1782 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.092056 2074 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:52.104833 2162 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:52.105482 2162 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:52.105825 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.106889 2162 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:52.107125 2173 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:52.114790 2129 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.114187 1564 catalog_manager.cc:5495] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 2 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } }
I20260619 21:15:52.116431 1739 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:52.141076 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.145439 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.146438 1663 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.156459 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.158540 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.162658 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.162835 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.156502 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.175184 1669 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:52.179189 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.180099 1739 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.179265 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:52.184783 1817 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:52.188764 1663 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:52.191329 1708 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:52.193018 2135 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:52.193650 2135 raft_consensus.cc:686] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:52.194900 2135 consensus_queue.cc:237] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.197290 1859 tablet_bootstrap.cc:654] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:52.197441 2015 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.199630 1668 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:52.207664 1860 tablet_bootstrap.cc:492] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:52.208298 1860 ts_tablet_manager.cc:1397] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.375s user 0.020s sys 0.030s
I20260619 21:15:52.209530 2162 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.210574 1667 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:52.180649 2129 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.210965 1563 catalog_manager.cc:5495] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:52.227579 1782 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.229833 2114 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:52.230242 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.231640 1782 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.237135 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:52.242936 1635 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.232613 2162 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.246225 1860 raft_consensus.cc:348] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.247212 1860 raft_consensus.cc:374] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.247537 1860 raft_consensus.cc:729] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.244895 2114 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:52.250062 2114 raft_consensus.cc:686] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.251616 1809 raft_consensus.cc:1261] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.248383 1860 consensus_queue.cc:260] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.256209 1860 ts_tablet_manager.cc:1428] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.046s user 0.008s sys 0.000s
I20260619 21:15:52.258216 1858 tablet_bootstrap.cc:492] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:52.258666 1858 ts_tablet_manager.cc:1397] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.346s user 0.015s sys 0.008s
I20260619 21:15:52.262563 1860 tablet_bootstrap.cc:492] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:52.242573 1665 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:52.259624 1739 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:52.251570 2114 consensus_queue.cc:237] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.265057 1634 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:52.266724 2015 raft_consensus.cc:2781] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:52.267652 2153 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.269585 2153 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.261915 2135 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260619 21:15:52.269589 2015 raft_consensus.cc:686] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:52.276264 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:52.276168 2015 consensus_queue.cc:237] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.278221 2153 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.289870 1739 raft_consensus.cc:2454] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:52.291352 1634 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:52.307842 2153 raft_consensus.cc:2781] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:52.262519 1782 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.285250 1666 raft_consensus.cc:2454] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:52.309257 2153 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:52.313581 2162 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:52.315871 2162 raft_consensus.cc:686] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.316847 2162 consensus_queue.cc:237] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.312692 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:52.314400 2153 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:52.320389 1635 leader_election.cc:341] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.321156 1859 tablet_bootstrap.cc:492] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:52.321723 1859 ts_tablet_manager.cc:1397] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.276s user 0.015s sys 0.039s
I20260619 21:15:52.325219 1859 raft_consensus.cc:348] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.325891 1859 raft_consensus.cc:374] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.326143 1859 raft_consensus.cc:729] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.326681 1859 consensus_queue.cc:260] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.328858 1859 ts_tablet_manager.cc:1428] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260619 21:15:52.330018 1859 tablet_bootstrap.cc:492] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:52.333068 1858 raft_consensus.cc:348] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.334967 1858 raft_consensus.cc:374] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.335316 1858 raft_consensus.cc:729] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.336205 1858 consensus_queue.cc:260] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.342788 1858 ts_tablet_manager.cc:1428] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.084s user 0.007s sys 0.001s
I20260619 21:15:52.346196 1858 tablet_bootstrap.cc:492] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:52.351758 1860 tablet_bootstrap.cc:654] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:52.314659 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 53706.8us
W20260619 21:15:52.361711 2191 log.cc:924] Time spent T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Append to log took a long time: real 0.082s user 0.002s sys 0.000s
I20260619 21:15:52.361625 1782 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.363080 2074 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:52.364899 2074 raft_consensus.cc:686] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.365978 2015 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.372500 2015 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.371035 2074 consensus_queue.cc:237] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.365126 1782 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.379806 1782 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.381963 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:52.384647 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.385203 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:52.386289 1859 tablet_bootstrap.cc:654] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
W20260619 21:15:52.385469 1635 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:52.387739 1634 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.387872 2164 raft_consensus.cc:2781] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:52.387961 2173 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.388737 2164 raft_consensus.cc:686] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.389138 2173 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.390316 1634 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.382752 1666 raft_consensus.cc:1261] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.391383 2017 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:52.392410 2173 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.390041 2164 consensus_queue.cc:237] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.395233 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:52.396085 1738 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:52.379801 2162 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:52.401544 2139 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:52.402374 1783 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:52.404800 2173 raft_consensus.cc:2781] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:52.405333 2173 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:52.405782 2173 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.407284 1858 tablet_bootstrap.cc:654] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:52.391091 2015 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.415715 1817 raft_consensus.cc:1261] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.402658 2162 raft_consensus.cc:686] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.417722 2135 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260619 21:15:52.408219 1635 consensus_peers.cc:487] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:52.442232 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:52.418670 2173 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.420042 2153 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.420320 2162 consensus_queue.cc:237] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.453730 1810 raft_consensus.cc:1261] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.461009 2139 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260619 21:15:52.491695 2173 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.493444 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.494297 1736 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.494760 2153 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.461597 1782 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:52.496150 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 44275.9us
I20260619 21:15:52.496608 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.497229 2074 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:52.497365 1739 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:52.497637 1782 leader_election.cc:341] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.501598 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:52.502978 1635 leader_election.cc:341] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.502868 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:15:52.512714 1782 leader_election.cc:341] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.527115 1663 raft_consensus.cc:1261] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.528733 2139 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:52.531864 1860 tablet_bootstrap.cc:492] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:52.532351 1860 ts_tablet_manager.cc:1397] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.271s user 0.023s sys 0.036s
I20260619 21:15:52.534952 1860 raft_consensus.cc:348] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.535699 1860 raft_consensus.cc:374] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.535977 1860 raft_consensus.cc:729] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.536646 1860 consensus_queue.cc:260] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.538771 1736 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:52.526589 1739 raft_consensus.cc:2454] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:52.540360 1783 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:52.541396 2173 raft_consensus.cc:2781] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:52.541448 1634 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:52.542382 1860 ts_tablet_manager.cc:1428] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260619 21:15:52.543660 2153 raft_consensus.cc:2781] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:52.545363 2153 raft_consensus.cc:686] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:52.542378 2173 raft_consensus.cc:686] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.546283 2153 consensus_queue.cc:237] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.547731 2173 consensus_queue.cc:237] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.550689 1860 tablet_bootstrap.cc:492] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:52.559731 1817 raft_consensus.cc:1226] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:52.557888 1660 raft_consensus.cc:1261] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.602984 2132 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:52.564277 2135 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.604828 2074 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.605336 2074 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.605260 2135 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:52.636790 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 71910.9us
I20260619 21:15:52.566164 2139 consensus_queue.cc:1046] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260619 21:15:52.636375 2162 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.664362 1859 tablet_bootstrap.cc:492] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:52.653831 2015 consensus_queue.cc:1035] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260619 21:15:52.665146 1859 ts_tablet_manager.cc:1397] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.335s user 0.017s sys 0.029s
I20260619 21:15:52.666038 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:52.666155 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:52.667058 1660 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
W20260619 21:15:52.667961 2208 log.cc:924] Time spent T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.128s user 0.001s sys 0.003s
W20260619 21:15:52.671030 1708 leader_election.cc:341] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.674198 2135 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.675637 2135 raft_consensus.cc:3032] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:52.675953 2135 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:52.676687 2135 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.677618 1860 tablet_bootstrap.cc:654] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:52.678088 2135 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:52.660845 2162 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.681200 2162 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.684141 2074 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.689440 1858 tablet_bootstrap.cc:492] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:52.689947 1858 ts_tablet_manager.cc:1397] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.344s user 0.013s sys 0.007s
I20260619 21:15:52.698242 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:52.700151 1782 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:52.699031 1736 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:15:52.711081 1635 consensus_peers.cc:487] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:52.725551 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:52.726702 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:52.727699 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:52.728574 1666 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:52.708684 2074 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:52.730100 2074 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:52.730470 2074 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:52.732239 1783 leader_election.cc:341] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.749315 1782 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:52.751935 2162 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.717469 1565 catalog_manager.cc:5495] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 2 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:52.763692 2074 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.779102 2074 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:52.767261 1565 catalog_manager.cc:5495] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 2 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:52.788182 1859 raft_consensus.cc:348] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.789336 1859 raft_consensus.cc:374] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.789711 1859 raft_consensus.cc:729] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.785230 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:52.788167 1565 catalog_manager.cc:5495] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 2 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:52.790366 1859 consensus_queue.cc:260] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.795558 1859 ts_tablet_manager.cc:1428] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.130s user 0.009s sys 0.001s
I20260619 21:15:52.797168 1666 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:52.798561 1667 raft_consensus.cc:1261] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.799923 2173 consensus_queue.cc:1035] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:52.796458 1859 tablet_bootstrap.cc:492] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:52.805030 2162 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260619 21:15:52.814354 1710 leader_election.cc:341] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.815013 1710 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:52.814723 1666 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:52.820081 1782 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:52.820786 2164 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:52.821269 2164 raft_consensus.cc:686] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:52.824782 1858 raft_consensus.cc:348] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.825606 1858 raft_consensus.cc:374] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.825919 1858 raft_consensus.cc:729] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.826665 1858 consensus_queue.cc:260] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.829507 2164 consensus_queue.cc:237] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:52.830521 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:52.816388 2135 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:52.837961 1859 tablet_bootstrap.cc:654] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:52.850486 1858 ts_tablet_manager.cc:1428] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.160s user 0.011s sys 0.019s
I20260619 21:15:52.850678 2217 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.851554 1858 tablet_bootstrap.cc:492] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:52.851214 2217 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.853158 2212 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:52.853837 1666 raft_consensus.cc:1261] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.858261 2217 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.858122 2173 consensus_queue.cc:1035] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:52.866258 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:52.866930 1739 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:52.870433 1634 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:52.865087 2213 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260619 21:15:52.872857 2217 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:52.879606 1741 raft_consensus.cc:1261] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.871906 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:52.885592 1810 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:52.879874 2015 consensus_queue.cc:1035] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260619 21:15:52.886721 1665 raft_consensus.cc:1261] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.883455 2217 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:52.887990 2217 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:52.884496 1739 raft_consensus.cc:1261] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.888814 2162 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:52.894497 2212 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:52.894963 2212 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.906878 1860 tablet_bootstrap.cc:492] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:52.906885 2212 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.907580 1858 tablet_bootstrap.cc:654] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:52.907668 1860 ts_tablet_manager.cc:1397] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.357s user 0.033s sys 0.024s
I20260619 21:15:52.904474 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:52.914234 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:52.915580 1669 raft_consensus.cc:1261] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:52.921157 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:52.923142 1708 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:52.927325 2074 consensus_queue.cc:1035] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:52.930451 2213 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:52.920774 2212 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:15:52.932262 1783 leader_election.cc:341] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:52.932621 2212 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.938640 1738 raft_consensus.cc:3037] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:52.939457 2162 consensus_queue.cc:1035] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:52.927663 2217 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.938151 2125 consensus_queue.cc:1035] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260619 21:15:52.947861 2015 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.960395 2015 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.958267 1860 raft_consensus.cc:348] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:52.962657 1860 raft_consensus.cc:374] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:52.962993 1860 raft_consensus.cc:729] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:52.963430 2015 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.963796 1860 consensus_queue.cc:260] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:52.990850 2218 log.cc:924] Time spent T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
I20260619 21:15:52.990899 1860 ts_tablet_manager.cc:1428] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.083s user 0.006s sys 0.002s
I20260619 21:15:52.993193 2153 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:52.999737 2213 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:15:53.015061 2223 log.cc:924] Time spent T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.069s user 0.001s sys 0.000s
I20260619 21:15:53.006003 2212 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:52.999418 2017 consensus_queue.cc:1035] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260619 21:15:53.033260 2173 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:53.012305 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 78450.9us
I20260619 21:15:53.057842 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:53.013792 2153 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.055310 1858 tablet_bootstrap.cc:492] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:53.065177 1858 ts_tablet_manager.cc:1397] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.214s user 0.018s sys 0.012s
I20260619 21:15:53.054764 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:53.056852 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:53.019892 2213 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:53.067536 2221 log.cc:924] Time spent T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.142s user 0.001s sys 0.000s
I20260619 21:15:53.083426 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:53.084069 1816 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:15:53.084614 2224 log.cc:924] Time spent T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.085s user 0.001s sys 0.000s
I20260619 21:15:53.088200 1738 raft_consensus.cc:1261] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:53.089100 1810 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:53.069429 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:53.090366 1735 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:53.043470 2173 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.093252 1634 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:53.094252 1859 tablet_bootstrap.cc:492] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:53.097671 1859 ts_tablet_manager.cc:1397] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.301s user 0.016s sys 0.009s
I20260619 21:15:53.079928 1741 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:53.108520 1816 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:15:53.116387 1860 tablet_bootstrap.cc:492] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:53.078238 2217 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:53.117501 2015 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:53.129858 2213 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:53.126775 1858 raft_consensus.cc:348] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.131218 1858 raft_consensus.cc:374] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:53.132728 1858 raft_consensus.cc:729] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:53.121989 2015 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:53.146935 2217 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:53.147536 2217 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.147919 2153 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:53.133463 1858 consensus_queue.cc:260] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.149607 2217 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:53.131407 2173 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:53.370100 1741 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:15:53.378772 1858 ts_tablet_manager.cc:1428] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.313s user 0.010s sys 0.012s
I20260619 21:15:53.379797 1858 tablet_bootstrap.cc:492] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:53.151082 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:53.371734 1859 raft_consensus.cc:348] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.456593 1859 raft_consensus.cc:374] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:53.456907 1859 raft_consensus.cc:729] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:53.457608 1859 consensus_queue.cc:260] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:53.370589 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 274711us
I20260619 21:15:53.154500 2015 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:53.477705 1859 ts_tablet_manager.cc:1428] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.380s user 0.006s sys 0.017s
W20260619 21:15:53.557891 1781 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.041s sys 0.062s
I20260619 21:15:53.595935 2139 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:53.609611 1739 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:53.621603 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:53.624596 1740 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:53.625939 2164 consensus_queue.cc:1035] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:53.617491 2074 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:53.597182 2132 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:53.588641 2175 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:53.665203 1708 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.025s sys 0.043s
I20260619 21:15:53.667325 1635 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:15:53.667938 1859 ts_tablet_manager.cc:1460] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Tablet startup took 569ms
W20260619 21:15:53.955017 1859 ts_tablet_manager.cc:1462] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Trace:
0619 21:15:42.404404 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:42.732902 (+328498us) service_pool.cc:227] Handling call
0619 21:15:42.761169 (+ 28267us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:15:42.783446 (+ 22277us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:15:42.963332 (+179886us) tablet_metadata.cc:641] Metadata flushed
0619 21:15:43.101736 (+138404us) tablet_replica.cc:205] Creating consensus instance
0619 21:15:43.143912 (+ 42176us) inbound_call.cc:173] Queueing success response
0619 21:15:52.796150 (+9652238us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:15:53.098081 (+301931us) ts_tablet_manager.cc:1429] Starting tablet replica
0619 21:15:53.200912 (+102831us) tablet_replica.cc:259] Starting instrumentation
0619 21:15:53.201540 (+ 628us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":644,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":31227,"tablet-open.queue_time_us":9652337}
I20260619 21:15:53.955844 2246 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:53.956805 1859 tablet_bootstrap.cc:492] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:53.957324 2246 raft_consensus.cc:686] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:53.675122 1735 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
W20260619 21:15:53.667860 1781 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.628s user 0.041s sys 0.063s
W20260619 21:15:53.668507 1782 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.569s user 0.066s sys 0.159s
W20260619 21:15:54.641662 1782 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.542s user 0.068s sys 0.159s
W20260619 21:15:54.641968 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 1.54234e+06us
I20260619 21:15:53.958387 2246 consensus_queue.cc:237] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:53.953948 1708 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.860s user 0.029s sys 0.043s
W20260619 21:15:54.643265 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 1.5495e+06us
W20260619 21:15:53.683341 1737 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.007s sys 0.031s
W20260619 21:15:54.645119 1737 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.548s user 0.016s sys 0.031s
W20260619 21:15:53.959436 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 292212us
I20260619 21:15:53.966817 2247 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:53.891299 2015 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.652520 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:53.677412 2074 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.655747 1737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 198) took 1594 ms. Trace:
I20260619 21:15:54.656489 2074 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:53.678682 2132 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:53.855064 2199 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.720s user 0.080s sys 0.109s
W20260619 21:15:54.660411 2199 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 1.526s user 0.081s sys 0.111s
W20260619 21:15:53.850906 2216 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.697s user 0.018s sys 0.036s
I20260619 21:15:53.855271 1858 tablet_bootstrap.cc:654] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:53.834335 1860 tablet_bootstrap.cc:654] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:53.672222 2139 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.887779 2217 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:53.678426 2175 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.690686 2234 consensus_queue.cc:1035] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.152s
I20260619 21:15:53.983465 2248 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:54.693369 2248 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:53.965152 1859 tablet_bootstrap.cc:654] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:53.991395 2249 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.697310 2217 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.517409 2250 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:54.518630 1632 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.526s user 0.049s sys 0.084s
I20260619 21:15:54.519695 2252 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.520366 2251 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.521054 2129 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:54.529009 2254 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.539125 2176 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.539732 2253 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.546568 2255 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.550344 2256 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.551445 2257 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.553649 2258 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.563076 2173 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.564069 2260 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.564330 2238 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:15:54.567178 1739 consensus_meta.cc:287] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.896s user 0.014s sys 0.005s
I20260619 21:15:54.568089 2259 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.569988 2237 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.573702 2261 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.574651 2262 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:15:54.579912 1741 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.964s user 0.039s sys 0.095s
W20260619 21:15:54.580624 1710 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.99.3:45591 timed out after 1.577s (SENT) [suppressed 6 similar messages]
I20260619 21:15:54.580861 2164 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.590792 2263 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:15:54.602368 1740 consensus_meta.cc:287] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.929s user 0.014s sys 0.007s
I20260619 21:15:54.610846 2235 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.620114 1565 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37726 (request call id 94) took 2474 ms. Trace:
I20260619 21:15:54.621487 2265 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.623023 2266 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:54.624063 1783 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:54.624208 2267 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:15:54.630616 1736 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.014s user 0.009s sys 0.030s
I20260619 21:15:54.633783 2240 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.634243 1564 catalog_manager.cc:5495] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:54.635828 2125 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.637171 2162 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:54.638123 2230 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:54.650522 2247 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.653587 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:54.658910 2132 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.660619 2015 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.656114 1737 rpcz_store.cc:276] 0619 21:15:53.061049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:53.063895 (+ 2846us) service_pool.cc:227] Handling call
0619 21:15:53.072516 (+ 8621us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:15:54.645488 (+1572972us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:15:54.645520 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:15:54.645934 (+ 414us) log.cc:842] Serialized 26 byte log entry
0619 21:15:54.655383 (+ 9449us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:15:54.655425 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:15:54.655493 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:15:54.655534 (+ 41us) raft_consensus.cc:1671] finished
0619 21:15:54.655551 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:15:54.655664 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10231,"raft.queue_time_us":47614,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4184,"thread_start_us":1562106,"threads_started":2,"wal-append.queue_time_us":1035}
I20260619 21:15:54.643081 2268 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:15:54.669745 2216 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.515s user 0.020s sys 0.036s
W20260619 21:15:54.707567 1632 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.715s user 0.050s sys 0.084s
I20260619 21:15:54.697717 2249 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.706588 2213 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:54.698784 2250 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:54.725306 1741 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.110s user 0.039s sys 0.096s
I20260619 21:15:54.772282 1564 catalog_manager.cc:5495] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:54.723762 2175 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.706331 2129 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:54.763679 1736 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.147s user 0.010s sys 0.030s
I20260619 21:15:54.708112 2176 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.808769 1740 raft_consensus.cc:1261] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:54.716766 2252 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.805483 2074 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.774250 1739 raft_consensus.cc:1261] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:54.757717 1565 rpcz_store.cc:276] 0619 21:15:52.146047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:52.146228 (+ 181us) service_pool.cc:227] Handling call
0619 21:15:54.620034 (+2473806us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:15:52.878755 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:15:52.879317 (+ 562us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:15:52.879375 (+ 58us) write_op.cc:435] Acquired schema lock
0619 21:15:52.879392 (+ 17us) tablet.cc:660] Decoding operations
0619 21:15:52.897552 (+ 18160us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:15:52.903383 (+ 5831us) write_op.cc:641] Partition lock acquired for write op
0619 21:15:52.903726 (+ 343us) tablet.cc:683] Acquiring locks for 3 operations
0619 21:15:52.917307 (+ 13581us) tablet.cc:699] Row locks acquired
0619 21:15:52.917328 (+ 21us) write_op.cc:260] PREPARE: finished
0619 21:15:52.917470 (+ 142us) write_op.cc:270] Start()
0619 21:15:52.917584 (+ 114us) write_op.cc:276] Timestamp: P: 1781903752917434 usec, L: 0
0619 21:15:52.917605 (+ 21us) op_driver.cc:331] REPLICATION: starting
0619 21:15:52.918216 (+ 611us) log.cc:842] Serialized 1289 byte log entry
0619 21:15:53.133587 (+215371us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b5400152fa0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43af0 00007f719ea4e028 00007f719ea4dc6a 00007f719eab9cd5 00007f719eaba179 00007f71a24cdade 00007f71a24cd2b8 00007f71a24d1ba1 00007f71a24d1b59 00007f71a24d1ae9 00007f71a24d1ab1
0619 21:15:53.133837 (+ 250us) op_driver.cc:444] REPLICATION: finished
0619 21:15:53.889847 (+756010us) write_op.cc:301] APPLY: starting
0619 21:15:53.890648 (+ 801us) tablet.cc:1365] starting BulkCheckPresence
0619 21:15:53.891554 (+ 906us) tablet.cc:1368] finished BulkCheckPresence
0619 21:15:53.891571 (+ 17us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:15:53.892183 (+ 612us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:15:53.892425 (+ 242us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0619 21:15:53.892475 (+ 50us) write_op.cc:312] APPLY: finished
0619 21:15:53.892766 (+ 291us) log.cc:842] Serialized 34 byte log entry
0619 21:15:53.893590 (+ 824us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:53.893682 (+ 92us) write_op.cc:454] Released schema lock
0619 21:15:53.893710 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0619 21:15:53.894157 (+ 447us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:53.894185 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9272,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":755729,"apply.run_cpu_time_us":2150,"apply.run_wall_time_us":4623,"num_ops":3,"prepare.queue_time_us":50011,"prepare.run_cpu_time_us":7551,"prepare.run_wall_time_us":255045,"replication_time_us":216132,"spinlock_wait_cycles":12561280,"thread_start_us":1553724,"threads_started":2}]]}
I20260619 21:15:54.829419 2280 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.756227 2263 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.846724 2273 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.847688 2274 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.804884 2125 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.757726 2259 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.723292 2251 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.804630 2240 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.716300 2173 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.758534 2164 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.757313 2238 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.813336 2015 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:54.831379 2139 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.725131 2254 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.749284 2257 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.751358 2260 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.755244 2256 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.820340 1817 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:54.716326 2253 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.761564 2261 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.764604 2237 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.848919 2249 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.805015 2162 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.866972 2234 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260619 21:15:54.879118 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:54.867309 2241 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:54.831581 2213 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.821048 2230 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.863885 2248 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.868993 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 187) took 1256 ms. Trace:
I20260619 21:15:54.874101 2279 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:54.895334 2125 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:54.896387 2260 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.899475 2256 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.751542 2258 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.875698 1739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 179) took 1280 ms. Trace:
I20260619 21:15:54.750382 2255 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.859563 2074 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.879369 1741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 178) took 1284 ms. Trace:
I20260619 21:15:54.883381 2217 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.878780 2280 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.889370 2236 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:54.884385 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 180) took 1288 ms. Trace:
I20260619 21:15:54.807457 2267 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.771773 2265 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.761386 2262 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.867388 2274 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.849751 2268 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.907845 2234 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:54.886684 2015 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.805480 2235 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.910416 2247 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.807642 2266 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.921530 2279 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.944675 2254 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.866986 2273 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.948140 2273 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.920233 1740 rpcz_store.cc:276] 0619 21:15:53.612229 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:53.620592 (+ 8363us) service_pool.cc:227] Handling call
0619 21:15:53.624476 (+ 3884us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:15:54.850641 (+1226165us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:15:54.864414 (+ 13773us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:15:54.951153 2235 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:54.936120 2164 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:54.953490 2252 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.929744 2234 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:54.927675 1739 rpcz_store.cc:276] 0619 21:15:53.595568 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:53.609060 (+ 13492us) service_pool.cc:227] Handling call
0619 21:15:53.609452 (+ 392us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:15:54.857433 (+1247981us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:15:54.873863 (+ 16430us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:15:54.965368 2240 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:54.965672 2280 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.967378 2258 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.973608 2266 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:54.975397 2263 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.928551 2241 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:54.995709 2277 log.cc:924] Time spent T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.138s user 0.000s sys 0.001s
I20260619 21:15:54.998201 2251 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.999146 2015 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.001192 2129 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.999046 2262 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:55.002954 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 422284us
I20260619 21:15:54.987896 2259 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:54.958498 2236 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.997722 2173 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.008836 2213 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.022568 2235 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:55.025466 2250 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.036530 2274 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.038065 2162 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.043774 2074 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.033089 2125 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:55.036806 2257 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.069952 2274 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.065791 2213 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:55.085371 2139 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:55.053542 2285 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:55.061642 2261 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.066843 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.101323 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.085107 2129 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:54.930619 1736 rpcz_store.cc:276] 0619 21:15:53.595917 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:53.597033 (+ 1116us) service_pool.cc:227] Handling call
0619 21:15:53.606003 (+ 8970us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:15:53.616005 (+ 10002us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:15:53.616034 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:15:53.616435 (+ 401us) log.cc:842] Serialized 26 byte log entry
0619 21:15:54.857579 (+1241144us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:15:54.860978 (+ 3399us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:15:54.868212 (+ 7234us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:15:54.873284 (+ 5072us) raft_consensus.cc:1671] finished
0619 21:15:54.875339 (+ 2055us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:15:54.882069 (+ 6730us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9516,"raft.queue_time_us":10415,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4331,"thread_start_us":1227389,"threads_started":1,"wal-append.queue_time_us":266158,"wal-append.run_cpu_time_us":1498,"wal-append.run_wall_time_us":1001533}
I20260619 21:15:55.130580 2129 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.131965 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:55.085599 2212 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:55.133023 1738 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:55.088351 2250 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.063680 2235 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.089154 2259 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.142169 1669 raft_consensus.cc:2379] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:55.095563 2139 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.133117 2212 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:55.136463 1839 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1280 after lost signal to thread 399
I20260619 21:15:55.097429 2176 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:55.086270 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 462231us
I20260619 21:15:55.182847 2236 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.185850 2267 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.184144 2237 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.128253 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.087786 2125 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.192394 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.199190 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.202015 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.202886 2251 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:55.203495 2263 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:55.204121 2258 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.097244 2213 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.096513 2274 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.164269 2261 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.171142 2216 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.171049 2230 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.170529 2262 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.167927 2257 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:55.168663 2250 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.168962 2259 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:54.926887 1741 rpcz_store.cc:276] 0619 21:15:53.595217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:53.596694 (+ 1477us) service_pool.cc:227] Handling call
0619 21:15:53.605578 (+ 8884us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:15:53.614896 (+ 9318us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:15:53.614923 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:15:53.615385 (+ 462us) log.cc:842] Serialized 26 byte log entry
0619 21:15:54.849714 (+1234329us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:15:54.856967 (+ 7253us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:15:54.860612 (+ 3645us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:15:54.863034 (+ 2422us) raft_consensus.cc:1671] finished
0619 21:15:54.867854 (+ 4820us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:15:54.877791 (+ 9937us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8874,"raft.queue_time_us":9074,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":144,"thread_start_us":1214952,"threads_started":1,"wal-append.queue_time_us":245302,"wal-append.run_cpu_time_us":1836,"wal-append.run_wall_time_us":1014450}
I20260619 21:15:55.222855 2132 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.267951 2263 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.268523 2255 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.176635 2253 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.253086 1818 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:55.213544 1739 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.095654 2285 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:55.353607 1635 leader_election.cc:341] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:55.252127 2251 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:55.371385 1635 consensus_peers.cc:487] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260619 21:15:55.371588 1859 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.672s user 0.102s sys 0.173s
W20260619 21:15:55.373003 1859 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.674s user 0.103s sys 0.173s
I20260619 21:15:55.374074 1635 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:55.375452 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.375985 1669 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:55.377499 1669 raft_consensus.cc:1261] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260619 21:15:55.354187 1564 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s user 0.017s sys 0.026s
W20260619 21:15:55.381632 1564 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.528s user 0.024s sys 0.026s
W20260619 21:15:55.386729 1782 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.659s user 0.010s sys 0.040s
W20260619 21:15:55.387086 1782 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.659s user 0.010s sys 0.040s
W20260619 21:15:55.327828 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 64723.5us
I20260619 21:15:55.281628 2286 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:55.327970 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.327113 2213 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.321272 2237 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.282505 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.320250 2129 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.345350 2265 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:15:55.389200 1839 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1700 after lost signal to thread 1601
I20260619 21:15:55.349426 2268 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.276473 2258 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.286347 2262 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.316825 2257 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.388350 2175 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:55.396181 2015 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:55.279253 2261 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.395877 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.350503 2132 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.416499 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.410735 1735 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
W20260619 21:15:55.426443 2283 log.cc:924] Time spent T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.336s user 0.001s sys 0.000s
I20260619 21:15:55.398984 2279 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.411482 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.411302 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.429171 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.403446 2237 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.411082 2238 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:55.437050 2289 log.cc:924] Time spent T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.151s user 0.001s sys 0.001s
I20260619 21:15:55.436743 2251 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.460026 1782 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:55.432274 2175 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.440182 2293 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:55.281975 2216 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.464594 1858 tablet_bootstrap.cc:492] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:55.468447 1858 ts_tablet_manager.cc:1397] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 2.089s user 0.017s sys 0.009s
I20260619 21:15:55.444644 2235 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:15:55.457033 1839 debug-util.cc:398] Leaking SignalData structure 0x7b080029ad60 after lost signal to thread 1693
I20260619 21:15:55.435173 2015 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.450842 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.451669 2212 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.449559 2139 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.463943 2125 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.477654 1860 tablet_bootstrap.cc:492] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:55.478071 1860 ts_tablet_manager.cc:1397] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 2.375s user 0.025s sys 0.029s
I20260619 21:15:55.410811 2286 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.479712 1859 tablet_bootstrap.cc:492] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:55.480046 1859 ts_tablet_manager.cc:1397] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 1.523s user 0.109s sys 0.178s
I20260619 21:15:55.468724 2257 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:55.470089 1839 debug-util.cc:398] Leaking SignalData structure 0x7b080029ad40 after lost signal to thread 1694
I20260619 21:15:55.461405 2114 consensus_queue.cc:1035] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260619 21:15:55.473919 2293 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:55.481280 2241 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.485657 2293 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:55.485502 2250 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.491035 2263 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.493945 2259 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.495497 2261 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.496738 2286 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.493379 2285 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.531307 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.550791 2015 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:55.533684 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 479952us
I20260619 21:15:55.551369 2175 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.554008 2262 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.554576 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.555966 1665 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:55.542724 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.557441 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.557983 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.558396 1665 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:55.558627 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.559163 1739 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.559561 2132 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.559664 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.560241 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:55.536475 1707 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.706s user 0.029s sys 0.039s
I20260619 21:15:55.537303 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.538906 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:55.539130 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 79065.8us
W20260619 21:15:55.455536 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 53943.7us
I20260619 21:15:55.541680 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.542073 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:55.548286 1839 debug-util.cc:398] Leaking SignalData structure 0x7b0800296b80 after lost signal to thread 1768
I20260619 21:15:55.561761 2074 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260619 21:15:55.536201 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 211504us
I20260619 21:15:55.581229 1743 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.584146 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.583200 2216 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.585140 1741 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.585286 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.583045 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.586361 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:55.585311 1839 debug-util.cc:398] Leaking SignalData structure 0x7b0800296bc0 after lost signal to thread 1769
I20260619 21:15:55.580669 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.595192 1664 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
W20260619 21:15:55.595520 2303 log.cc:924] Time spent T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Append to log took a long time: real 0.096s user 0.001s sys 0.000s
I20260619 21:15:55.594296 1743 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.579249 1814 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.598907 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.600323 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.601186 1740 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:55.601994 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.602175 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.602761 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.603967 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.605330 1743 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.605820 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.607077 1737 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
W20260619 21:15:55.609551 1839 debug-util.cc:398] Leaking SignalData structure 0x7b0800278960 after lost signal to thread 1842
W20260619 21:15:55.610025 1839 debug-util.cc:398] Leaking SignalData structure 0x7b0800278980 after lost signal to thread 1843
I20260619 21:15:55.617193 1810 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.618335 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.618623 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.599437 1859 raft_consensus.cc:348] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.600528 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:55.579973 1707 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.765s user 0.032s sys 0.042s
I20260619 21:15:55.620142 1859 raft_consensus.cc:374] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:55.622576 1859 raft_consensus.cc:729] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:55.619336 1810 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.558418 2237 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.562912 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.602809 1735 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.587035 2074 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:55.620994 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.631932 2074 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:55.632354 1741 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.624722 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.633404 1667 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:55.633164 1858 raft_consensus.cc:348] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.634017 1858 raft_consensus.cc:374] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:55.634286 1858 raft_consensus.cc:729] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:55.634541 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.635372 1667 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:55.634968 1858 consensus_queue.cc:260] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.640400 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:55.641111 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.641371 1812 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:55.641983 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.642704 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.644341 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.652031 1634 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.653991 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.655328 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.659795 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.661751 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.662101 1737 raft_consensus.cc:1261] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:55.668324 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.669373 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.670042 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.663057 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.663669 2286 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:55.665333 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.630134 1859 consensus_queue.cc:260] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:55.667543 1782 consensus_peers.cc:487] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 -> Peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Couldn't send request to peer 884c214bb0a7455d91701ec388635e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:55.630623 1860 raft_consensus.cc:348] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.667673 1741 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.674021 1860 raft_consensus.cc:374] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:55.652873 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.674971 1740 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:55.675793 1860 raft_consensus.cc:729] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:55.670258 2268 consensus_queue.cc:1035] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260619 21:15:55.648828 2274 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.663672 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.686172 2286 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.686858 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.687431 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.682767 1860 consensus_queue.cc:260] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.667699 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.670738 1736 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:55.688556 1710 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:55.693517 1810 raft_consensus.cc:2379] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:55.695335 2261 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:55.695814 2261 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:55.696159 2261 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:55.696252 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.684041 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.705454 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.737205 2230 consensus_queue.cc:1035] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:55.737982 1634 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.705646 1859 ts_tablet_manager.cc:1428] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.225s user 0.010s sys 0.000s
I20260619 21:15:55.692378 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.702010 1735 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.741305 1663 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:55.741227 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.742740 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.745682 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.694836 1809 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:55.736469 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.746502 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:55.757165 1738 consensus_meta.cc:287] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.622s user 0.020s sys 0.001s
I20260619 21:15:55.696751 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.753685 1858 ts_tablet_manager.cc:1428] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.285s user 0.000s sys 0.016s
I20260619 21:15:55.757452 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.751965 2237 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:55.736479 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 48229.2us
I20260619 21:15:55.754611 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.765662 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.768103 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.778045 1858 tablet_bootstrap.cc:492] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:55.781425 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.781816 1664 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:55.783070 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.783674 1811 raft_consensus.cc:1261] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:55.761274 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:55.785562 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 97248.3us
I20260619 21:15:55.785645 1741 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.786736 1859 tablet_bootstrap.cc:492] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:55.785822 2274 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:55.776496 1812 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:15:55.789645 1860 ts_tablet_manager.cc:1428] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.311s user 0.009s sys 0.001s
I20260619 21:15:55.779385 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.778909 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.775795 2258 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.790977 2274 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260619 21:15:55.802580 1708 leader_election.cc:341] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:55.809638 2274 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:15:55.809814 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 45702.2us
I20260619 21:15:55.803141 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.763494 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.843039 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.770423 1738 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
W20260619 21:15:55.810746 1710 leader_election.cc:341] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:55.837821 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.800261 1634 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.803779 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.839221 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.840088 1860 tablet_bootstrap.cc:492] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:55.844615 1739 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:55.845574 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.846704 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.848904 2261 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.784636 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.772226 2268 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:15:55.775803 2234 consensus_meta.cc:287] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.806s user 0.006s sys 0.023s
I20260619 21:15:55.775072 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.792101 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.802062 2286 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.869243 1708 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.875842 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.876807 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.877583 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.857560 1737 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.886178 2114 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:55.808390 2293 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:55.902321 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 113513us
I20260619 21:15:55.903020 2258 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:55.875407 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:55.922439 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 59784.8us
I20260619 21:15:55.795814 2265 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260619 21:15:55.903648 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.878289 1743 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:55.921685 1740 raft_consensus.cc:2379] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:55.934329 2074 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.904282 2114 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.937547 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.938689 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.939872 1858 tablet_bootstrap.cc:654] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:55.940023 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.941179 1816 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:55.938740 1660 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:55.942056 1662 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:55.943339 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.945030 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.948385 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.921422 2258 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:55.949532 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.949666 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.950090 2258 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:55.950518 1813 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:55.953433 1710 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:55.953650 2015 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:55.954196 2015 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.955160 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.870519 2234 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.956805 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.939594 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.956919 2015 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.958120 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:55.959896 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 173105us
I20260619 21:15:55.940796 1668 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:55.968182 1782 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:55.949539 1634 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.958133 1815 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:55.930512 2237 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:55.960070 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.959645 2129 raft_consensus.cc:2726] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:55.872982 2268 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:55.940737 2114 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:55.992349 1708 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.981312 1665 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:55.962119 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.958534 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.970937 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.943372 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:55.959779 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.959192 2234 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:15:56.004485 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 55096us
I20260619 21:15:55.959231 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.962124 2261 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:55.959292 1664 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:56.013149 1634 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:55.960829 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.013484 1662 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:56.014144 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:56.014039 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.015057 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:56.004678 2230 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:55.961076 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:55.988785 2237 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:55.967808 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:55.981791 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:56.004917 2015 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:55.995244 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.001736 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:56.023976 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 59445.1us
I20260619 21:15:56.031810 1818 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:56.032699 2129 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.029479 2237 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:56.025719 2230 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.029767 2074 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.046675 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.058125 1816 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:56.041435 2015 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.029562 1663 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:56.060669 1708 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.053602 2230 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:15:56.065275 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 77199.6us
I20260619 21:15:56.050115 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.051280 2129 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.074537 2236 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:56.074978 2236 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.075351 2236 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:56.074031 1782 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.062805 2015 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.085390 1708 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.069206 2261 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.077942 2129 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.103690 2265 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.104236 2265 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.094965 2293 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.104964 2132 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.096022 2261 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.118994 1860 tablet_bootstrap.cc:654] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:56.119815 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.104696 2265 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.122432 2132 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.124994 2132 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.102819 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.129108 1782 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.141132 1811 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
W20260619 21:15:56.175331 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 53451.9us
W20260619 21:15:56.175664 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 44866.3us
I20260619 21:15:56.122663 2261 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.190958 1782 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.191152 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:56.217207 2285 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.217849 2285 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.218288 2285 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.222846 1634 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.224057 2286 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.224205 1634 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.224598 2286 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.225082 2286 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.225432 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:56.146886 1742 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:15:56.228677 2114 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.252455 2114 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.253039 2114 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:15:56.278920 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 48185us
I20260619 21:15:56.273119 2293 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.273784 1738 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:56.289388 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 65278.7us
I20260619 21:15:56.285281 2268 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.288875 2293 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:15:56.310027 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 227555us
I20260619 21:15:56.308058 1783 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.316695 2327 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.317538 2327 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.324928 1634 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.328444 1635 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.309825 2293 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.333210 1782 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:15:56.340108 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 164705us
I20260619 21:15:56.342285 1783 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.358649 1782 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.336261 2268 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.370432 1858 tablet_bootstrap.cc:492] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:56.336488 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 216) took 1032 ms. Trace:
I20260619 21:15:56.344048 2327 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.374938 1858 ts_tablet_manager.cc:1397] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.597s user 0.014s sys 0.018s
I20260619 21:15:56.366947 2234 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:56.375752 2274 consensus_meta.cc:287] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.509s user 0.016s sys 0.002s
I20260619 21:15:56.401012 2340 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.404631 2263 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:56.418638 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 102721us
W20260619 21:15:56.419641 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 116544us
W20260619 21:15:56.438647 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 59269.6us
I20260619 21:15:56.421962 2340 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.419760 2074 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:56.423190 2274 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.448009 1634 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:15:56.462278 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 241633us
I20260619 21:15:56.464195 2263 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.464634 2263 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.465044 2263 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.468163 1634 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.454806 2290 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.447954 2340 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.475088 1635 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.491504 1635 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.466302 2129 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.491911 2290 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260619 21:15:56.508131 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 60217.3us
W20260619 21:15:56.515014 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 46968.9us
W20260619 21:15:56.515468 1710 leader_election.cc:341] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.516197 2331 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.517199 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.517921 1811 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.510047 2290 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.522331 2129 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.524109 2015 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.524344 2236 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.525383 1662 raft_consensus.cc:1226] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:56.527050 2015 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.527475 2236 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.529078 2337 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260619 21:15:56.530668 2317 log.cc:924] Time spent T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.359s user 0.003s sys 0.000s
I20260619 21:15:56.533672 2274 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:56.505892 2258 consensus_meta.cc:287] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.555s user 0.018s sys 0.007s
I20260619 21:15:56.540688 2132 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.541944 2236 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:56.542938 2132 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.543675 1736 raft_consensus.cc:2354] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:15:56.542642 2236 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.545099 2129 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.545640 2129 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.546782 2236 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.547468 2129 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:56.548341 1739 consensus_meta.cc:287] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.681s user 0.012s sys 0.008s
I20260619 21:15:56.550606 2274 consensus_queue.cc:1035] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:56.551769 2230 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.553421 2334 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.371052 1666 rpcz_store.cc:276] 0619 21:15:55.197124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:55.995370 (+798246us) service_pool.cc:227] Handling call
0619 21:15:56.098271 (+102901us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:15:56.110721 (+ 12450us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:15:56.130863 (+ 20142us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:15:56.142366 (+ 11503us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:15:56.168160 (+ 25794us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:15:56.192591 (+ 24431us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:15:56.204755 (+ 12164us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:15:56.229635 (+ 24880us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":30706560}
I20260619 21:15:56.554224 1634 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.555343 2286 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.561748 2334 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.562150 2334 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.561992 1858 raft_consensus.cc:348] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.562534 2334 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.562742 1858 raft_consensus.cc:374] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:56.563011 1858 raft_consensus.cc:729] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:56.563663 1858 consensus_queue.cc:260] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.565223 1635 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.565874 2258 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.536811 1811 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:15:56.572239 1858 ts_tablet_manager.cc:1428] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.188s user 0.007s sys 0.000s
I20260619 21:15:56.573463 1634 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.575378 1564 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37710 (request call id 79) took 3670 ms. Trace:
W20260619 21:15:56.575611 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 62079.3us
W20260619 21:15:56.576489 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 66118.5us
I20260619 21:15:56.575093 1859 tablet_bootstrap.cc:654] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:56.575981 1669 raft_consensus.cc:1226] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260619 21:15:56.579991 1710 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.597559 2324 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.598160 2324 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.598604 2324 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.578182 2286 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.575419 2274 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.600126 2274 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.600510 2274 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.592793 2337 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:56.577392 1563 catalog_manager.cc:5495] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:56.605285 2337 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.605417 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.605684 2337 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:56.606480 1635 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.609174 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.574791 2258 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.613822 1783 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.611913 1563 catalog_manager.cc:5495] T ef644e16cf5b46b68fde33b180432159 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:56.575547 2340 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.611536 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.619102 1635 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.619184 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.620354 1635 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.620537 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.619443 2340 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.616662 1738 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:56.577703 2261 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.594369 2293 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.630278 1858 tablet_bootstrap.cc:492] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:56.576242 2285 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.629092 1563 catalog_manager.cc:5495] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:56.626538 2342 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.622517 2286 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.627394 2015 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.628799 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:56.635658 1710 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.635407 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.576606 2290 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.636503 1635 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:56.677380 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 63663.1us
I20260619 21:15:56.627643 2258 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.631428 2230 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:15:56.695098 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 64017.3us
I20260619 21:15:56.681589 1634 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.684316 1563 catalog_manager.cc:5495] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:56.684180 2015 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.703352 2015 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.703867 2340 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.708261 2274 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.709087 2341 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.709589 2341 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.709970 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.710482 2274 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.711624 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.711911 2341 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.715197 1811 raft_consensus.cc:2379] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:56.715312 2290 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.714984 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:56.716722 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 260922us
I20260619 21:15:56.719094 1783 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.719475 1708 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.684207 2286 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:15:56.724213 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 87773.5us
W20260619 21:15:56.718575 2237 consensus_meta.cc:287] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.650s user 0.009s sys 0.011s
I20260619 21:15:56.728365 1783 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.728453 1708 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.696025 1815 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:56.727837 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.729504 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.729947 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.730351 1738 raft_consensus.cc:2379] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:56.731123 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.731446 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.731666 1815 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.731761 2285 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.725036 2286 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.732664 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.733278 1813 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:15:56.726089 2265 consensus_meta.cc:287] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.529s user 0.009s sys 0.002s
I20260619 21:15:56.737291 2346 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.737406 2265 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.737826 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.739496 2265 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.739868 1859 tablet_bootstrap.cc:492] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:56.730284 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.740363 1859 ts_tablet_manager.cc:1397] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.954s user 0.030s sys 0.057s
I20260619 21:15:56.740525 1809 raft_consensus.cc:1226] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:56.741779 2261 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.737846 2346 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.742702 2346 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.742641 2263 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.719980 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.697736 2114 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.721966 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.747231 1635 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.748708 2349 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.746492 2268 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.749941 2293 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.749985 2114 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.585381 1564 rpcz_store.cc:276] 0619 21:15:52.905154 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:52.906216 (+ 1062us) service_pool.cc:227] Handling call
0619 21:15:56.575098 (+3668882us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:15:55.437504 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:15:55.483651 (+ 46147us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:15:55.484632 (+ 981us) write_op.cc:435] Acquired schema lock
0619 21:15:55.485384 (+ 752us) tablet.cc:660] Decoding operations
0619 21:15:55.497131 (+ 11747us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:15:55.497207 (+ 76us) write_op.cc:641] Partition lock acquired for write op
0619 21:15:55.497225 (+ 18us) tablet.cc:683] Acquiring locks for 2 operations
0619 21:15:55.497486 (+ 261us) tablet.cc:699] Row locks acquired
0619 21:15:55.497497 (+ 11us) write_op.cc:260] PREPARE: finished
0619 21:15:55.497676 (+ 179us) write_op.cc:270] Start()
0619 21:15:55.497776 (+ 100us) write_op.cc:276] Timestamp: P: 1781903755497630 usec, L: 0
0619 21:15:55.497792 (+ 16us) op_driver.cc:331] REPLICATION: starting
0619 21:15:55.498511 (+ 719us) log.cc:842] Serialized 910 byte log entry
0619 21:15:55.756267 (+257756us) op_driver.cc:444] REPLICATION: finished
0619 21:15:56.311200 (+554933us) write_op.cc:301] APPLY: starting
0619 21:15:56.326469 (+ 15269us) tablet.cc:1365] starting BulkCheckPresence
0619 21:15:56.364492 (+ 38023us) tablet.cc:1368] finished BulkCheckPresence
0619 21:15:56.409198 (+ 44706us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:15:56.409743 (+ 545us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:15:56.410067 (+ 324us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0619 21:15:56.410153 (+ 86us) write_op.cc:312] APPLY: finished
0619 21:15:56.410522 (+ 369us) log.cc:842] Serialized 28 byte log entry
0619 21:15:56.410756 (+ 234us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:56.410901 (+ 145us) write_op.cc:454] Released schema lock
0619 21:15:56.410971 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0619 21:15:56.411320 (+ 349us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:56.411355 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":538140,"apply.run_cpu_time_us":3528,"apply.run_wall_time_us":108047,"num_ops":2,"prepare.queue_time_us":551091,"prepare.run_cpu_time_us":4987,"prepare.run_wall_time_us":176409,"raft.queue_time_us":157051,"raft.run_cpu_time_us":11474,"raft.run_wall_time_us":79873,"replication_time_us":258286,"thread_start_us":700601,"threads_started":4,"wal-append.queue_time_us":996}]]}
I20260619 21:15:56.752056 1860 tablet_bootstrap.cc:492] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:56.736199 2237 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.754426 2334 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.747985 1710 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.762713 1860 ts_tablet_manager.cc:1397] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.923s user 0.023s sys 0.034s
I20260619 21:15:56.762291 1708 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.766631 1783 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.754818 2015 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.755489 1563 catalog_manager.cc:5495] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 2, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 2 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:56.777240 1708 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.746091 2274 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:56.777138 2334 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.778154 2327 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:56.778884 2285 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:56.778748 1563 catalog_manager.cc:5495] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:56.780108 1859 raft_consensus.cc:348] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.780395 2328 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:56.780731 1710 leader_election.cc:341] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.724574 2341 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.781272 1708 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.764358 2265 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.782819 2327 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:15:56.782887 1710 leader_election.cc:341] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.782958 2265 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.783450 1710 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.770462 1858 tablet_bootstrap.cc:654] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:56.744906 2263 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.771937 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.785418 2327 raft_consensus.cc:2726] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.786149 2265 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.772707 2286 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.764734 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:56.775434 2337 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.776361 2349 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.764837 2237 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.776857 2261 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.780807 1859 raft_consensus.cc:374] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:56.790994 1859 raft_consensus.cc:729] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:56.791705 1859 consensus_queue.cc:260] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.792414 2337 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.793732 2015 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.782100 2285 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:15:56.782966 1635 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.784159 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.796327 1710 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.784726 1783 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:56.760978 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 122280us
I20260619 21:15:56.795148 1815 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:15:56.781289 2274 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.797642 2285 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.804647 1859 ts_tablet_manager.cc:1428] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.064s user 0.002s sys 0.006s
I20260619 21:15:56.802632 2274 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:56.806308 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.806553 2285 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.803758 1860 raft_consensus.cc:348] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.807543 1708 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.808379 2265 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.808691 1783 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.797796 2324 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.811333 1635 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.814936 2261 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:56.815419 2324 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.812795 2015 consensus_queue.cc:1035] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260619 21:15:56.816099 2265 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.816370 1635 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.812157 1860 raft_consensus.cc:374] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:56.817243 1860 raft_consensus.cc:729] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:56.818106 1860 consensus_queue.cc:260] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.813936 1859 tablet_bootstrap.cc:492] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:56.811587 2337 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:56.821890 1635 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.804549 1813 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:56.808482 2346 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.822747 2337 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:56.823449 2337 raft_consensus.cc:686] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:56.820619 1860 ts_tablet_manager.cc:1428] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.054s user 0.004s sys 0.003s
I20260619 21:15:56.824168 1635 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.813946 1783 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.824925 1860 tablet_bootstrap.cc:492] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:56.821444 2261 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.825991 1783 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.826733 2285 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.826987 2114 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:15:56.827482 2114 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.827948 2114 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:56.828500 2346 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.810501 2286 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.830706 2237 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.831097 2237 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.831444 2237 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:15:56.833648 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 45527.9us
I20260619 21:15:56.834671 1710 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.836040 1708 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.824831 2337 consensus_queue.cc:237] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.841988 2237 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.846290 1635 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.849411 2015 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.849900 2015 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:56.849653 1635 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.850569 2324 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.847637 2346 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.847469 2327 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.852737 2349 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.853066 2261 raft_consensus.cc:2726] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.858379 2346 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.858762 2346 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:15:56.858918 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:56.859568 2261 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.850384 2015 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.861208 2237 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.864066 1634 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.865077 1783 leader_election.cc:398] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:15:56.865402 1783 leader_election.cc:401] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:15:56.865927 2285 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:15:56.866899 2285 consensus_queue.cc:1035] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:56.868445 2015 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.869588 1710 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.870366 2015 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.870426 2328 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260619 21:15:56.871964 1710 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.875252 1783 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.876724 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.877625 1660 raft_consensus.cc:2379] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:56.877133 2263 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.878154 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:56.876921 1634 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.878314 2263 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.877530 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.896111 1669 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:56.898620 1634 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:56.901074 2334 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:56.901237 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.854071 2114 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.905988 1710 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.894315 2263 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.907382 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.908192 2324 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:56.908563 1710 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:56.905543 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:56.911581 2293 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.909744 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.903744 2285 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:56.911866 2324 raft_consensus.cc:686] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:56.915627 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.877184 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:56.916471 1661 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:56.915947 2324 consensus_queue.cc:237] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.898514 2286 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:56.917327 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.918159 1815 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:56.877310 2328 consensus_queue.cc:1046] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260619 21:15:56.903455 2334 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:56.919687 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.920346 1669 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.920382 2285 raft_consensus.cc:686] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:56.921237 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.921339 2285 consensus_queue.cc:237] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.923718 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:56.924085 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.924444 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:56.923743 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:56.925424 2339 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:56.925343 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:56.925772 1744 raft_consensus.cc:2379] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:56.919929 2334 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:56.924918 1813 raft_consensus.cc:2379] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:56.926927 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:15:56.913553 1635 leader_election.cc:341] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.929375 1635 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.926283 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:56.944629 2333 raft_consensus.cc:3032] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:56.945048 2261 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:56.945194 2333 raft_consensus.cc:729] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:56.945590 2261 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:56.945987 2261 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.946352 2333 consensus_queue.cc:260] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.948304 2333 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:56.917268 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:56.925673 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.926314 1662 raft_consensus.cc:2379] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
W20260619 21:15:56.952642 1710 leader_election.cc:341] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:56.958778 2339 raft_consensus.cc:686] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:56.963140 1635 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:56.967449 2114 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:56.969594 1858 tablet_bootstrap.cc:492] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:56.970067 1858 ts_tablet_manager.cc:1397] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.340s user 0.022s sys 0.008s
I20260619 21:15:56.968046 2339 consensus_queue.cc:237] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.970758 2286 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:56.975081 1635 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:56.976238 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.977550 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:56.974749 2346 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.987088 1736 raft_consensus.cc:1226] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:56.989705 2346 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:56.990023 1740 raft_consensus.cc:1226] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:56.992646 1810 raft_consensus.cc:1226] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:15:56.950179 2274 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:56.995591 2340 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:57.000348 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.000586 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.001402 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.002175 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.002939 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.016310 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.016601 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.016985 1735 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:57.017180 1818 raft_consensus.cc:2354] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:15:57.020016 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.021278 2340 raft_consensus.cc:686] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:57.008379 1669 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:15:57.005157 1859 tablet_bootstrap.cc:654] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.015447 1817 raft_consensus.cc:1226] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.003666 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.021369 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.025729 1667 raft_consensus.cc:1226] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.021271 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.028189 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.029348 1661 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:57.029773 2237 consensus_queue.cc:1046] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:57.030349 1813 raft_consensus.cc:2379] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.030790 2346 raft_consensus.cc:3032] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.030908 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.015488 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.031188 2346 raft_consensus.cc:729] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.031791 1665 raft_consensus.cc:2354] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:15:57.016508 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.032351 2346 consensus_queue.cc:260] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.037355 1660 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:57.037614 2346 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.037981 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.038884 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.039180 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.039705 1738 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.039981 1812 raft_consensus.cc:2379] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:57.037020 2265 consensus_queue.cc:1046] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.041309 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.041460 1816 raft_consensus.cc:1226] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:15:57.041509 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.032490 2290 consensus_queue.cc:1046] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260619 21:15:57.042650 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.042785 2334 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.043490 2290 raft_consensus.cc:3032] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.043851 2290 raft_consensus.cc:729] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.022784 2340 consensus_queue.cc:237] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.044690 2290 consensus_queue.cc:260] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.045281 2274 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.045629 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.045869 2290 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.046108 2334 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.046386 1810 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.051425 2129 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:57.051891 2129 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.053982 2129 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.057683 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:57.073326 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.073839 1668 raft_consensus.cc:1226] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.074819 1662 raft_consensus.cc:1226] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
W20260619 21:15:57.074880 1710 leader_election.cc:341] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.075395 1667 raft_consensus.cc:1226] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.076169 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.077097 1736 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:57.034497 2338 consensus_queue.cc:1046] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260619 21:15:57.083742 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.086972 1860 tablet_bootstrap.cc:654] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.072741 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.088455 1744 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:15:57.088758 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.089380 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.089959 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.090519 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.091131 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.091981 1666 raft_consensus.cc:1226] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:15:57.091912 2342 consensus_queue.cc:1046] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.092216 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.068008 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.098923 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.109225 2346 raft_consensus.cc:3032] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:57.109764 2346 raft_consensus.cc:729] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.109822 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.110049 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.110787 1737 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.110831 2346 consensus_queue.cc:260] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.111780 1660 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.104444 1742 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.121731 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.112126 2267 raft_consensus.cc:3032] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.125543 2267 raft_consensus.cc:729] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.123739 1661 raft_consensus.cc:1226] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.125982 1814 raft_consensus.cc:1226] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.123513 1634 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.127821 1634 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.128952 2334 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.130661 2267 consensus_queue.cc:260] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.093523 2114 raft_consensus.cc:3032] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:57.134114 1634 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.135723 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.136147 2015 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.137369 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.137497 1708 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.138316 1634 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.134433 2334 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.138055 1669 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:57.140146 2015 raft_consensus.cc:2781] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:57.140497 2015 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:57.111876 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.134634 2114 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.141048 2015 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.141390 1663 raft_consensus.cc:2379] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:57.139603 2267 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.141673 2114 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.143056 2114 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:57.143661 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.123034 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.145026 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.143826 2338 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.140456 1708 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.146024 1743 raft_consensus.cc:2354] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:15:57.148432 2326 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.148908 1708 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.147079 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.152153 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.152490 1660 raft_consensus.cc:2379] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:57.153967 2015 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.155134 2339 raft_consensus.cc:2726] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.157876 2015 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.157773 1858 raft_consensus.cc:348] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.159070 1858 raft_consensus.cc:374] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.159487 1858 raft_consensus.cc:729] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
W20260619 21:15:57.159164 2354 log.cc:924] Time spent T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.095s user 0.001s sys 0.000s
I20260619 21:15:57.145157 2324 consensus_queue.cc:1046] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:57.161578 2324 raft_consensus.cc:3032] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.161950 2324 raft_consensus.cc:729] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.162070 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.161684 1858 consensus_queue.cc:260] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.162849 2324 consensus_queue.cc:260] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.163429 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.165231 1858 ts_tablet_manager.cc:1428] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.195s user 0.006s sys 0.003s
I20260619 21:15:57.156059 1708 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.162734 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.171118 2339 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.157791 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.172129 1663 raft_consensus.cc:2379] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.173448 2346 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:57.165977 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.179231 1858 tablet_bootstrap.cc:492] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:57.179805 1742 raft_consensus.cc:2379] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.180100 1708 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.182235 2326 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.182468 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.156543 2261 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.187132 2261 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.187427 1708 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.190636 2349 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.191146 1735 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:15:57.191869 1741 raft_consensus.cc:1226] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:15:57.193517 1783 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:57.194411 2342 raft_consensus.cc:2781] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:15:57.195511 2342 raft_consensus.cc:686] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.196638 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.196456 2342 consensus_queue.cc:237] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.198698 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.200770 1708 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.206126 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.209595 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.211053 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.211236 2333 raft_consensus.cc:2726] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.213200 2263 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.214339 1660 raft_consensus.cc:1226] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.215358 2263 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.215766 1782 leader_election.cc:398] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:15:57.198845 2324 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.219180 2290 consensus_queue.cc:1046] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260619 21:15:57.222369 2234 consensus_queue.cc:1046] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:57.218685 2364 consensus_queue.cc:1046] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.221118 2230 consensus_queue.cc:1046] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260619 21:15:57.227906 2230 raft_consensus.cc:3032] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.228394 2230 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.228716 2290 raft_consensus.cc:3032] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.229269 2290 raft_consensus.cc:729] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.229308 2074 raft_consensus.cc:3032] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:57.229794 2074 raft_consensus.cc:729] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.230309 2290 consensus_queue.cc:260] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.230633 2074 consensus_queue.cc:260] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.232013 2074 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:57.231957 2290 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.233593 2364 raft_consensus.cc:3032] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.238497 2364 raft_consensus.cc:729] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.239506 2364 consensus_queue.cc:260] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.243747 2364 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.218178 2349 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:57.245318 1782 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.246131 1708 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.230248 2230 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.251616 1563 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37734 (request call id 78) took 4449 ms. Trace:
I20260619 21:15:57.252745 1782 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.251801 1708 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.258456 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.254052 1635 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.252076 2129 raft_consensus.cc:2781] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:57.252671 2349 raft_consensus.cc:686] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.261462 2263 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.260104 1782 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.270970 2234 raft_consensus.cc:2726] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.272251 1859 tablet_bootstrap.cc:492] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:57.262737 1635 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.272814 1859 ts_tablet_manager.cc:1397] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.459s user 0.018s sys 0.007s
I20260619 21:15:57.273097 2263 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.251236 2285 consensus_queue.cc:1046] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260619 21:15:57.274652 2285 raft_consensus.cc:3032] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.275070 2285 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.255975 1563 rpcz_store.cc:276] 0619 21:15:52.793715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:52.793937 (+ 222us) service_pool.cc:227] Handling call
0619 21:15:57.242129 (+4448192us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:15:56.909749 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:15:56.911390 (+ 1641us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:15:56.911449 (+ 59us) write_op.cc:435] Acquired schema lock
0619 21:15:56.911468 (+ 19us) tablet.cc:660] Decoding operations
0619 21:15:56.926095 (+ 14627us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:15:56.926210 (+ 115us) write_op.cc:641] Partition lock acquired for write op
0619 21:15:56.926241 (+ 31us) tablet.cc:683] Acquiring locks for 6 operations
0619 21:15:56.926909 (+ 668us) tablet.cc:699] Row locks acquired
0619 21:15:56.926950 (+ 41us) write_op.cc:260] PREPARE: finished
0619 21:15:56.927123 (+ 173us) write_op.cc:270] Start()
0619 21:15:56.927269 (+ 146us) write_op.cc:276] Timestamp: P: 1781903756927072 usec, L: 0
0619 21:15:56.927292 (+ 23us) op_driver.cc:331] REPLICATION: starting
0619 21:15:56.961274 (+ 33982us) log.cc:842] Serialized 2427 byte log entry
0619 21:15:57.067663 (+106389us) op_driver.cc:444] REPLICATION: finished
0619 21:15:57.165890 (+ 98227us) write_op.cc:301] APPLY: starting
0619 21:15:57.172897 (+ 7007us) tablet.cc:1365] starting BulkCheckPresence
0619 21:15:57.182304 (+ 9407us) tablet.cc:1368] finished BulkCheckPresence
0619 21:15:57.182439 (+ 135us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:15:57.183406 (+ 967us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:15:57.183643 (+ 237us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0619 21:15:57.183698 (+ 55us) write_op.cc:312] APPLY: finished
0619 21:15:57.184019 (+ 321us) log.cc:842] Serialized 52 byte log entry
0619 21:15:57.184211 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:57.184342 (+ 131us) write_op.cc:454] Released schema lock
0619 21:15:57.184388 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0619 21:15:57.184670 (+ 282us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:57.184705 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":74111,"apply.run_cpu_time_us":3240,"apply.run_wall_time_us":48698,"num_ops":6,"prepare.queue_time_us":102941,"prepare.run_cpu_time_us":4680,"prepare.run_wall_time_us":54789,"replication_time_us":138334,"thread_start_us":84602,"threads_started":2}]]}
I20260619 21:15:57.275862 2285 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.242240 2114 consensus_queue.cc:1046] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260619 21:15:57.276167 1859 raft_consensus.cc:348] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.250514 2265 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:57.277276 2114 raft_consensus.cc:3032] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:15:57.277705 2114 raft_consensus.cc:729] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.277412 2265 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.278777 2114 consensus_queue.cc:260] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.276897 1859 raft_consensus.cc:374] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.260793 2341 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.250522 2333 consensus_queue.cc:1046] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260619 21:15:57.265290 2129 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:57.255298 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.265936 1565 catalog_manager.cc:5495] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:57.269028 1744 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:57.269433 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.260653 2268 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.260910 2339 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.264616 1860 tablet_bootstrap.cc:492] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:57.285295 1860 ts_tablet_manager.cc:1397] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.461s user 0.017s sys 0.007s
I20260619 21:15:57.272203 1782 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.281277 2265 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:57.256609 2267 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.283972 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.285996 1634 leader_election.cc:398] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:15:57.282745 2129 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.286226 1741 raft_consensus.cc:2454] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:15:57.287623 1783 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:57.288388 1660 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:15:57.288499 2341 raft_consensus.cc:2781] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:57.288906 2341 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:57.289335 2341 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.286309 1634 leader_election.cc:401] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:15:57.291231 2015 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:15:57.291694 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.280546 1859 raft_consensus.cc:729] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.283631 2114 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:57.294299 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.295526 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.282756 2261 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:57.299470 2285 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.297230 2349 consensus_queue.cc:237] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.297473 2261 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.285985 1635 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.306560 1782 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.308984 1634 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:57.307796 2015 consensus_queue.cc:1046] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260619 21:15:57.310534 2015 raft_consensus.cc:3032] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:15:57.310849 2015 raft_consensus.cc:729] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:57.311479 2015 consensus_queue.cc:260] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.312541 2015 raft_consensus.cc:3037] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:57.313118 2290 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.307070 2326 raft_consensus.cc:3032] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.313678 2326 raft_consensus.cc:729] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.307524 2261 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.306797 1859 consensus_queue.cc:260] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.316393 1858 tablet_bootstrap.cc:654] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.314533 2326 consensus_queue.cc:260] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.317773 2326 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.322129 1859 ts_tablet_manager.cc:1428] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260619 21:15:57.323196 1859 tablet_bootstrap.cc:492] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:57.325273 1565 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37726 (request call id 95) took 1184 ms. Trace:
I20260619 21:15:57.298305 2230 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.326924 2290 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.325814 2263 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.327919 2261 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:57.331059 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.331732 2261 raft_consensus.cc:686] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:57.328238 1635 leader_election.cc:398] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:15:57.333590 1860 raft_consensus.cc:348] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.328809 1782 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.334411 1860 raft_consensus.cc:374] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.334762 1860 raft_consensus.cc:729] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.335306 1782 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:57.332540 1634 leader_election.cc:341] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.334210 1635 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.335896 1860 consensus_queue.cc:260] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.326591 1669 raft_consensus.cc:1226] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:15:57.337792 2263 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.325875 1565 rpcz_store.cc:276] 0619 21:15:56.141528 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:56.141938 (+ 410us) service_pool.cc:227] Handling call
0619 21:15:57.325178 (+1183240us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:15:57.297087 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:15:57.297609 (+ 522us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:15:57.297641 (+ 32us) write_op.cc:435] Acquired schema lock
0619 21:15:57.297660 (+ 19us) tablet.cc:660] Decoding operations
0619 21:15:57.297974 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:15:57.301040 (+ 3066us) write_op.cc:641] Partition lock acquired for write op
0619 21:15:57.301076 (+ 36us) tablet.cc:683] Acquiring locks for 1 operations
0619 21:15:57.301282 (+ 206us) tablet.cc:699] Row locks acquired
0619 21:15:57.301300 (+ 18us) write_op.cc:260] PREPARE: finished
0619 21:15:57.301454 (+ 154us) write_op.cc:270] Start()
0619 21:15:57.301557 (+ 103us) write_op.cc:276] Timestamp: P: 1781903757301416 usec, L: 0
0619 21:15:57.301577 (+ 20us) op_driver.cc:331] REPLICATION: starting
0619 21:15:57.302167 (+ 590us) log.cc:842] Serialized 531 byte log entry
0619 21:15:57.303963 (+ 1796us) op_driver.cc:444] REPLICATION: finished
0619 21:15:57.304408 (+ 445us) write_op.cc:301] APPLY: starting
0619 21:15:57.304483 (+ 75us) tablet.cc:1365] starting BulkCheckPresence
0619 21:15:57.304605 (+ 122us) tablet.cc:1368] finished BulkCheckPresence
0619 21:15:57.304704 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:15:57.304968 (+ 264us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:15:57.305139 (+ 171us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0619 21:15:57.305188 (+ 49us) write_op.cc:312] APPLY: finished
0619 21:15:57.305407 (+ 219us) log.cc:842] Serialized 22 byte log entry
0619 21:15:57.308205 (+ 2798us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:57.308289 (+ 84us) write_op.cc:454] Released schema lock
0619 21:15:57.308328 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0619 21:15:57.308588 (+ 260us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:57.308620 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":224,"apply.run_cpu_time_us":1824,"apply.run_wall_time_us":5676,"num_ops":1,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":2347,"prepare.run_wall_time_us":5327,"replication_time_us":2287}]]}
I20260619 21:15:57.339439 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.339483 2267 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.305797 2346 consensus_queue.cc:1046] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260619 21:15:57.343820 1860 ts_tablet_manager.cc:1428] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.058s user 0.006s sys 0.002s
I20260619 21:15:57.344687 1782 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.345223 2263 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.345707 1635 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.347687 2267 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.339640 2265 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.348271 2263 raft_consensus.cc:2726] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.347679 2129 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.349654 1635 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.350117 1741 raft_consensus.cc:2379] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:57.350528 2263 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.350804 2324 raft_consensus.cc:3032] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:15:57.351980 1635 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:57.353596 1635 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.354148 1635 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.351199 2324 raft_consensus.cc:729] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.355523 2263 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.355571 2334 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.356477 1635 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.358116 1635 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.357375 2334 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.359102 2263 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.360379 1635 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.356323 2324 consensus_queue.cc:260] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.362644 2263 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.363639 2129 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.364492 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.365326 1669 raft_consensus.cc:2379] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:57.366456 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.367318 2324 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:57.367375 1810 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.370572 1635 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.371500 1741 raft_consensus.cc:1261] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260619 21:15:57.372483 1635 leader_election.cc:341] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.372879 1635 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.374514 2015 raft_consensus.cc:2726] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.377715 2334 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:15:57.382762 1860 tablet_bootstrap.cc:492] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:57.383498 1859 tablet_bootstrap.cc:654] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.385165 2340 consensus_queue.cc:1035] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:57.386605 1635 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:15:57.387751 1635 leader_election.cc:341] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.388200 1635 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.387761 2340 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:15:57.389027 2340 raft_consensus.cc:2726] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.389487 1669 raft_consensus.cc:1261] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.392294 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:57.393169 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:57.334610 2261 consensus_queue.cc:237] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.355237 2341 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.396306 1563 catalog_manager.cc:5495] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:57.397651 1858 tablet_bootstrap.cc:492] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:57.398083 1858 ts_tablet_manager.cc:1397] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.219s user 0.008s sys 0.009s
W20260619 21:15:57.395200 1782 leader_election.cc:341] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.398532 1563 catalog_manager.cc:5495] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:57.393602 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:57.406929 2341 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:57.407510 1814 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:57.407680 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:15:57.409731 1635 leader_election.cc:341] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.409884 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:57.410188 1635 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.411298 2015 raft_consensus.cc:2726] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.413522 1563 catalog_manager.cc:5495] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:57.415066 1563 catalog_manager.cc:5495] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } }
W20260619 21:15:57.421135 1782 leader_election.cc:341] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.422516 1635 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:57.406630 1810 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:57.394304 2129 consensus_queue.cc:1035] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260619 21:15:57.428056 1710 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:57.420800 1736 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.426100 2230 consensus_queue.cc:1035] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260619 21:15:57.430253 2340 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:57.444887 2340 raft_consensus.cc:686] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:57.449079 2349 raft_consensus.cc:2781] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:57.449609 2340 consensus_queue.cc:237] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.450376 1814 raft_consensus.cc:1261] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.451187 1810 raft_consensus.cc:1261] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.452239 2230 consensus_queue.cc:1035] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 21:15:57.453197 2349 raft_consensus.cc:686] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:57.454587 1860 tablet_bootstrap.cc:654] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.454183 2349 consensus_queue.cc:237] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.457442 2015 consensus_queue.cc:1035] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:57.462054 1858 raft_consensus.cc:348] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.463236 1858 raft_consensus.cc:374] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.463590 1858 raft_consensus.cc:729] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.464476 1858 consensus_queue.cc:260] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.462057 2261 consensus_queue.cc:1035] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260619 21:15:57.484506 1810 raft_consensus.cc:1261] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260619 21:15:57.484752 1744 raft_consensus.cc:1261] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.488194 2015 consensus_queue.cc:1035] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.506631 1736 raft_consensus.cc:2454] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:57.493223 2346 consensus_queue.cc:1035] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260619 21:15:57.507970 1858 ts_tablet_manager.cc:1428] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.110s user 0.007s sys 0.003s
I20260619 21:15:57.508268 1783 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:57.509038 1858 tablet_bootstrap.cc:492] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:57.510062 2230 raft_consensus.cc:2781] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:57.510596 2230 raft_consensus.cc:686] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:57.525861 1669 raft_consensus.cc:1261] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260619 21:15:57.532111 2129 consensus_queue.cc:1035] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260619 21:15:57.533689 2230 consensus_queue.cc:237] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.557691 1814 raft_consensus.cc:1261] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.562353 2326 consensus_queue.cc:1035] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.568835 1563 catalog_manager.cc:5495] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:57.586548 1563 catalog_manager.cc:5495] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 3 to 4, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 4 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:57.596015 1739 raft_consensus.cc:1261] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260619 21:15:57.602787 2334 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:57.603377 2334 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.605576 2334 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.602048 2341 consensus_queue.cc:1035] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260619 21:15:57.622674 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:57.623450 1739 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
W20260619 21:15:57.626590 2382 log.cc:924] Time spent T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.060s user 0.004s sys 0.000s
I20260619 21:15:57.626910 1860 tablet_bootstrap.cc:492] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:57.627383 1860 ts_tablet_manager.cc:1397] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.245s user 0.025s sys 0.005s
I20260619 21:15:57.629154 1634 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:57.631014 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:57.632241 1860 raft_consensus.cc:348] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.633755 2274 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:57.634310 2274 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:57.634696 2274 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:57.639209 1635 leader_election.cc:341] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.640599 1860 raft_consensus.cc:374] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.640976 1860 raft_consensus.cc:729] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.641615 1860 consensus_queue.cc:260] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.648497 1860 ts_tablet_manager.cc:1428] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.021s user 0.008s sys 0.000s
I20260619 21:15:57.649714 1860 tablet_bootstrap.cc:492] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:57.663432 2349 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:57.664212 2349 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.660044 2274 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.675046 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:15:57.678217 1708 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.679448 2349 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.680629 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:15:57.682509 1635 leader_election.cc:341] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.682384 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:57.683537 1742 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.688972 1860 tablet_bootstrap.cc:654] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.689560 1859 tablet_bootstrap.cc:492] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:57.678679 2274 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.692305 1858 tablet_bootstrap.cc:654] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.697644 1859 ts_tablet_manager.cc:1397] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.375s user 0.012s sys 0.006s
I20260619 21:15:57.699532 1565 catalog_manager.cc:5495] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 2, leader changed from b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3) to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:57.700857 1565 catalog_manager.cc:5495] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 3, leader changed from 884c214bb0a7455d91701ec388635e74 (127.0.99.1) to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 3 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } }
I20260619 21:15:57.710531 1859 raft_consensus.cc:348] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.711385 1859 raft_consensus.cc:374] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.711670 1859 raft_consensus.cc:729] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.704872 1565 catalog_manager.cc:5495] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 2, leader changed from d53146739e3240239c0a3d5ea90e07fe (127.0.99.2) to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 2 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:57.721773 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:15:57.726531 1710 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:57.727072 1710 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.733033 1859 consensus_queue.cc:260] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.728648 2349 raft_consensus.cc:2726] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:57.742331 1663 raft_consensus.cc:1261] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260619 21:15:57.742959 1663 pending_rounds.cc:76] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: Aborting all ops after (but not including) 1
I20260619 21:15:57.743345 1663 pending_rounds.cc:97] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260619 21:15:57.743880 1663 raft_consensus.cc:2866] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:15:57.741420 1742 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:57.756645 1660 raft_consensus.cc:1261] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.780056 1634 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:57.783811 2334 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
W20260619 21:15:57.775771 2379 log.cc:924] Time spent T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.202s user 0.003s sys 0.000s
I20260619 21:15:57.800187 2230 consensus_queue.cc:1035] T 78cb98bb4e2642acb6448e5f5ccdb1ff P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.804216 1669 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:57.807657 1859 ts_tablet_manager.cc:1428] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.110s user 0.009s sys 0.000s
I20260619 21:15:57.758860 2349 consensus_queue.cc:1035] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260619 21:15:57.799183 2334 raft_consensus.cc:686] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:15:57.828910 1858 tablet_bootstrap.cc:492] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:57.829530 1858 ts_tablet_manager.cc:1397] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.321s user 0.045s sys 0.060s
I20260619 21:15:57.834646 1859 tablet_bootstrap.cc:492] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:57.835251 2334 consensus_queue.cc:237] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.855396 1858 raft_consensus.cc:348] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.856750 1858 raft_consensus.cc:374] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.857741 1816 raft_consensus.cc:1261] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:57.857092 1858 raft_consensus.cc:729] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.860700 2334 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:57.880615 1858 consensus_queue.cc:260] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.886078 1860 tablet_bootstrap.cc:492] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:57.886670 1860 ts_tablet_manager.cc:1397] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.237s user 0.015s sys 0.018s
I20260619 21:15:57.891263 1812 raft_consensus.cc:1261] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:15:57.899509 1859 tablet_bootstrap.cc:654] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.900183 2339 consensus_queue.cc:1035] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260619 21:15:57.901401 1860 raft_consensus.cc:348] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.904206 1860 raft_consensus.cc:374] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:57.904736 1860 raft_consensus.cc:729] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:57.908075 1858 ts_tablet_manager.cc:1428] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.078s user 0.007s sys 0.002s
I20260619 21:15:57.909046 1860 consensus_queue.cc:260] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.913409 1858 tablet_bootstrap.cc:492] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:57.920745 1860 ts_tablet_manager.cc:1428] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.034s user 0.009s sys 0.000s
I20260619 21:15:57.922256 1860 tablet_bootstrap.cc:492] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:57.932421 1742 raft_consensus.cc:1261] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:57.937268 2324 consensus_queue.cc:1035] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:57.939955 1858 tablet_bootstrap.cc:654] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:57.944674 1564 catalog_manager.cc:5495] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260619 21:15:57.954288 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:57.956084 2274 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:57.955145 2339 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:57.959831 2339 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:57.983628 1564 catalog_manager.cc:5495] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 2, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 2 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:15:57.968454 2339 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:57.999155 1564 catalog_manager.cc:5495] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:15:58.007530 1735 raft_consensus.cc:1261] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:58.009434 2334 consensus_queue.cc:1035] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:15:58.039410 1860 tablet_bootstrap.cc:654] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:58.043466 1810 raft_consensus.cc:1261] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:15:58.046663 2274 consensus_queue.cc:1035] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:15:58.054934 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.060304 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.069170 1664 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:15:58.072266 1708 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.076309 2326 raft_consensus.cc:2781] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:58.077353 2326 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:58.077823 2326 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:58.080574 1811 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:15:58.075593 1710 leader_election.cc:341] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:58.106592 1859 tablet_bootstrap.cc:492] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:58.107390 1859 ts_tablet_manager.cc:1397] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.282s user 0.014s sys 0.007s
I20260619 21:15:58.120448 1860 tablet_bootstrap.cc:492] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:58.121068 1860 ts_tablet_manager.cc:1397] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.199s user 0.018s sys 0.000s
I20260619 21:15:58.247089 2341 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.271900 2341 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:58.449870 1782 consensus_peers.cc:487] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 -> Peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Couldn't send request to peer 884c214bb0a7455d91701ec388635e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:58.450940 1859 raft_consensus.cc:348] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.451792 1859 raft_consensus.cc:374] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:58.452157 1859 raft_consensus.cc:729] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:58.452757 1859 consensus_queue.cc:260] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.458441 1860 raft_consensus.cc:348] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.468667 1860 raft_consensus.cc:374] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:58.469067 1860 raft_consensus.cc:729] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:58.475504 1859 ts_tablet_manager.cc:1428] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.368s user 0.013s sys 0.009s
I20260619 21:15:58.473726 1860 consensus_queue.cc:260] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.484105 2404 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.484604 1812 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:58.484925 1858 tablet_bootstrap.cc:492] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:58.485450 1858 ts_tablet_manager.cc:1397] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.572s user 0.011s sys 0.045s
I20260619 21:15:58.484633 2404 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.488227 1858 raft_consensus.cc:348] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.489619 1858 raft_consensus.cc:374] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:58.492559 1858 raft_consensus.cc:729] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:58.493867 1858 consensus_queue.cc:260] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.508860 1860 ts_tablet_manager.cc:1428] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.387s user 0.034s sys 0.020s
I20260619 21:15:58.519855 1859 tablet_bootstrap.cc:492] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:58.528558 1860 tablet_bootstrap.cc:492] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
W20260619 21:15:58.573160 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 66810.8us
I20260619 21:15:58.563201 2413 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.585842 1858 ts_tablet_manager.cc:1428] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.098s user 0.009s sys 0.010s
I20260619 21:15:58.585268 2265 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.590022 2324 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:58.576843 2326 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.593365 2263 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.602131 2285 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.610342 2129 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.617610 2418 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:58.585539 2413 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.618249 2263 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.613494 2265 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.589023 2407 consensus_queue.cc:1035] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260619 21:15:58.602313 2341 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.636075 1858 tablet_bootstrap.cc:492] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:58.629105 2261 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:58.638835 2404 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.630652 2324 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.671756 2263 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.630913 2285 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.632237 2129 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.654471 2261 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.653681 2416 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.658747 2423 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.675732 1860 tablet_bootstrap.cc:654] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:58.630348 2418 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.672890 1859 tablet_bootstrap.cc:654] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:58.692866 2404 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:58.677662 2326 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:58.708609 1635 consensus_peers.cc:487] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:15:58.674106 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.679158 2413 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.681632 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.684667 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:58.689335 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.688725 2324 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.704478 2411 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.702189 2341 consensus_queue.cc:1035] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260619 21:15:58.700862 2423 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:58.727298 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 63563.9us
I20260619 21:15:58.700699 2416 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.734441 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.734993 1743 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:58.725481 2413 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.737116 2413 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.739075 2413 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.743103 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.744220 1669 raft_consensus.cc:2379] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:58.745352 2425 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:58.745983 2129 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.745950 2425 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.748090 2425 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.748399 1858 tablet_bootstrap.cc:654] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:58.748756 2261 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.725863 2411 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.728051 1737 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:58.751190 2411 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.751529 2418 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.752038 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:58.752717 1660 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:58.754244 2346 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.754719 2346 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.724620 1740 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:58.714411 2404 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.728636 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.781797 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.779395 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.789561 1669 raft_consensus.cc:2379] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:58.793819 2349 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:58.796718 2349 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.775485 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.775949 2324 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.791467 2429 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:58.792207 2285 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.794222 1814 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:58.817026 2230 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:15:58.817276 2416 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.810576 1809 raft_consensus.cc:2379] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:58.820600 2346 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.817602 2230 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.823326 2349 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.824622 2230 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.809079 2324 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.814805 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.827927 2423 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.833212 1660 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:58.832429 1563 catalog_manager.cc:5495] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3). New cstate: current_term: 1 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260619 21:15:58.841769 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 76084.2us
I20260619 21:15:58.844110 1783 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.853114 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.842576 1738 raft_consensus.cc:2379] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:58.815749 2428 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:58.853961 1564 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37726 (request call id 96) took 1230 ms. Trace:
I20260619 21:15:58.853996 1669 raft_consensus.cc:2379] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:58.858238 1783 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.859338 1783 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.855938 1564 rpcz_store.cc:276] 0619 21:15:57.623185 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:57.623341 (+ 156us) service_pool.cc:227] Handling call
0619 21:15:58.853870 (+1230529us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:15:58.005336 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:15:58.005899 (+ 563us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:15:58.005936 (+ 37us) write_op.cc:435] Acquired schema lock
0619 21:15:58.005948 (+ 12us) tablet.cc:660] Decoding operations
0619 21:15:58.035954 (+ 30006us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:15:58.036013 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0619 21:15:58.036045 (+ 32us) tablet.cc:683] Acquiring locks for 3 operations
0619 21:15:58.061706 (+ 25661us) tablet.cc:699] Row locks acquired
0619 21:15:58.061725 (+ 19us) write_op.cc:260] PREPARE: finished
0619 21:15:58.061883 (+ 158us) write_op.cc:270] Start()
0619 21:15:58.062010 (+ 127us) write_op.cc:276] Timestamp: P: 1781903758061836 usec, L: 0
0619 21:15:58.062029 (+ 19us) op_driver.cc:331] REPLICATION: starting
0619 21:15:58.090862 (+ 28833us) log.cc:842] Serialized 1289 byte log entry
0619 21:15:58.217747 (+126885us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b180011dbe0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f71a24ce8a8 00007f71a24d0a78 00007f71a24d0a28 00007f71a24d09a8 00007f71a24d0963 00007f71a24cfc4c 00007f71a4ac7206 00007f71a4abf808 00007f719eaca170
0619 21:15:58.244761 (+ 27014us) op_driver.cc:444] REPLICATION: finished
0619 21:15:58.259747 (+ 14986us) write_op.cc:301] APPLY: starting
0619 21:15:58.261898 (+ 2151us) tablet.cc:1365] starting BulkCheckPresence
0619 21:15:58.293565 (+ 31667us) tablet.cc:1368] finished BulkCheckPresence
0619 21:15:58.295085 (+ 1520us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:15:58.340660 (+ 45575us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:15:58.340897 (+ 237us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0619 21:15:58.340991 (+ 94us) write_op.cc:312] APPLY: finished
0619 21:15:58.341307 (+ 316us) log.cc:842] Serialized 34 byte log entry
0619 21:15:58.341531 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:58.398547 (+ 57016us) write_op.cc:454] Released schema lock
0619 21:15:58.419526 (+ 20979us) write_op.cc:341] FINISH: Updating metrics
0619 21:15:58.453995 (+ 34469us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:15:58.455599 (+ 1604us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2499,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":4537,"apply.run_cpu_time_us":9682,"apply.run_wall_time_us":203070,"num_ops":3,"prepare.queue_time_us":100,"prepare.run_cpu_time_us":17650,"prepare.run_wall_time_us":88005,"replication_time_us":31283,"spinlock_wait_cycles":18227072}]]}
I20260619 21:15:58.866389 1708 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.850196 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.883878 1818 raft_consensus.cc:2379] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:15:58.863543 2285 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:15:58.884814 2285 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:58.885222 2285 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:58.835448 2429 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.844401 1663 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:58.873121 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.873306 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.873716 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.874189 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.863451 2428 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.874547 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.875059 2424 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:58.875254 1634 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:58.865168 2267 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:15:58.844738 2416 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:58.881464 2404 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.920662 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:15:58.897903 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 381134us
I20260619 21:15:58.891466 2429 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.904745 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.904925 1664 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:15:58.925817 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.926832 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.905543 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.927420 1660 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:58.928118 1738 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:15:58.929334 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.930116 1738 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:58.931213 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.931947 1738 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:58.904979 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.906446 2129 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.934196 1783 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.934155 2129 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.928149 1814 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:15:58.927536 1665 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:58.936631 2129 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.907990 2324 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:58.902040 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.938212 1669 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:58.879686 2265 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:58.918514 1708 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:15:58.963807 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 45415.9us
I20260619 21:15:58.902372 2423 raft_consensus.cc:2781] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:15:58.882893 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:58.898375 2411 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.905566 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.915849 2267 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:58.911908 2424 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:58.966303 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.966452 1708 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.980700 1669 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:58.920212 2409 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:15:58.981850 1710 leader_election.cc:341] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:58.983124 1710 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:58.924695 1739 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:58.919479 2408 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:58.919891 2425 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.899632 2427 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:58.981122 1708 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.922221 1662 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:58.988337 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.919787 1744 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:58.990780 1669 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:58.989290 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.991473 2425 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.992601 2129 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:58.993335 1708 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.993927 2324 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:58.995342 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:58.995852 2346 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:15:59.025941 2346 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.026019 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.026536 2346 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:59.027249 1664 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:58.995473 2427 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.997238 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.029474 1663 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:15:58.997859 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.030692 1817 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:59.031030 2427 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.031898 2417 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.032414 2417 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.033205 1811 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:59.035163 2324 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:58.983603 2265 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.035779 2324 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.035857 2265 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.036238 2265 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.039775 1860 tablet_bootstrap.cc:492] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:15:59.040262 1860 ts_tablet_manager.cc:1397] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.520s user 0.011s sys 0.025s
I20260619 21:15:59.041754 1708 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:58.998704 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.007259 1858 tablet_bootstrap.cc:492] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:15:58.969425 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.984800 2267 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:58.980628 2423 raft_consensus.cc:686] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:58.961975 2416 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.063318 2416 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:59.063971 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:15:59.064083 1743 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:15:58.983911 2424 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:58.984128 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.066987 1814 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
W20260619 21:15:59.016489 1710 leader_election.cc:341] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:15:59.067925 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 51481.3us
I20260619 21:15:58.989871 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:58.981300 2411 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.011152 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.070521 1740 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:59.071074 2411 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.003176 2230 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:59.071521 1666 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:15:59.071938 2230 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:58.987396 2409 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.073717 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.074641 1814 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:59.074914 2230 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.035102 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.043573 1858 ts_tablet_manager.cc:1397] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.417s user 0.016s sys 0.018s
I20260619 21:15:59.006167 2436 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.087110 2436 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.089098 1818 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:59.090857 2436 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.046783 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.022058 2129 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:59.049604 2265 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.094722 2129 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.096752 2265 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.019170 2334 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.035132 2428 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.097872 2334 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.032820 2417 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.101111 1667 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:59.019399 2407 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.041014 1665 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:59.054539 1810 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:15:59.104614 2407 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.106280 2407 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.108279 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.109355 2334 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.055718 2423 consensus_queue.cc:237] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.077919 1859 tablet_bootstrap.cc:492] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:15:59.078369 1710 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.117120 1859 ts_tablet_manager.cc:1397] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.637s user 0.029s sys 0.030s
I20260619 21:15:59.118944 1710 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:59.120410 1710 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:15:59.079587 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.123409 2407 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.123971 1811 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:59.082540 1708 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:59.124775 2428 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.125288 2428 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:59.125742 2428 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.127583 2326 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.128350 2326 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.129994 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.131225 1665 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:59.133167 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.128842 2326 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.134888 1634 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:59.135810 2334 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.136281 2334 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:59.136772 2334 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.137605 2274 consensus_queue.cc:1035] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:15:59.136734 1634 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:59.082918 2425 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.083200 2422 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.140720 2422 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.140849 2406 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.085053 1858 raft_consensus.cc:348] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.141229 2407 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.141371 2406 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:59.141702 1858 raft_consensus.cc:374] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:59.141983 2406 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.054695 1860 raft_consensus.cc:348] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.142539 1860 raft_consensus.cc:374] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:59.142853 1860 raft_consensus.cc:729] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:15:59.098245 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.143081 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.120734 2419 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.143640 1860 consensus_queue.cc:260] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.076239 1816 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:15:59.143922 1660 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:59.142035 1858 raft_consensus.cc:729] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:15:59.122756 2416 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.146605 2416 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:59.147116 2416 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.146862 1858 consensus_queue.cc:260] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.144178 1666 raft_consensus.cc:2379] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:59.144543 2419 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:59.141307 2422 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.119400 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.149003 2419 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.150933 1860 ts_tablet_manager.cc:1428] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.110s user 0.008s sys 0.001s
I20260619 21:15:59.152729 1860 tablet_bootstrap.cc:492] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:15:59.124264 1859 raft_consensus.cc:348] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.157943 1859 raft_consensus.cc:374] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:15:59.161595 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.162518 1742 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:59.163100 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.163743 1710 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.165091 1858 ts_tablet_manager.cc:1428] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.089s user 0.009s sys 0.000s
I20260619 21:15:59.165338 2425 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.165874 2425 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:15:59.166213 1858 tablet_bootstrap.cc:492] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:15:59.166390 2425 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.109246 1668 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:15:59.187150 2425 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.194442 2425 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.195351 2324 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:59.196298 1710 leader_election.cc:341] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:15:59.158241 1859 raft_consensus.cc:729] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:15:59.201040 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:59.198014 1859 consensus_queue.cc:260] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.202425 1669 raft_consensus.cc:2454] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
W20260619 21:15:59.205291 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 271127us
I20260619 21:15:59.206357 1783 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.218583 2267 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.222575 2267 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.226151 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.227121 1667 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.230274 1783 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.231580 1783 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.204195 2418 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.203094 2285 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.229267 2430 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:15:59.242130 1815 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:15:59.346853 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.348400 1740 raft_consensus.cc:2379] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:15:59.351810 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 117418us
I20260619 21:15:59.359364 1809 raft_consensus.cc:2354] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4.
I20260619 21:15:59.359930 1782 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:59.280510 2418 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.361460 2265 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:59.363013 1782 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:59.363641 2418 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.337024 2430 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:15:59.365604 2430 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.366637 2427 raft_consensus.cc:2726] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:59.370715 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.374771 1669 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:59.370879 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.319154 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.375667 1815 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:59.379560 1710 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.380474 1635 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:15:59.380749 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 172500us
I20260619 21:15:59.339161 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.349530 2341 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:15:59.389703 2421 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.394837 2421 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.395257 2421 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.353049 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.365625 2285 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.398018 2409 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.366899 2129 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.408362 1708 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:15:59.404518 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.354943 2446 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.394207 2341 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.407383 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.458251 1634 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.426191 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.351841 2416 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.378459 2428 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.441335 2346 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.425099 1663 raft_consensus.cc:2379] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:59.441959 2324 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.442672 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.443694 2449 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:15:59.447962 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:59.474645 1743 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.425264 2435 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.454103 1860 tablet_bootstrap.cc:654] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:59.457342 1667 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:15:59.452867 2446 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.458101 2429 raft_consensus.cc:2781] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:15:59.351572 2417 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:59.464586 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 52965.6us
I20260619 21:15:59.465651 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.477334 1859 ts_tablet_manager.cc:1428] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.360s user 0.015s sys 0.014s
I20260619 21:15:59.466555 1669 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:15:59.458343 2341 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:59.462496 2408 raft_consensus.cc:2726] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:15:59.448614 1812 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:59.493283 1782 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:59.493603 1783 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:59.496027 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.496809 1667 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.469897 1668 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:15:59.449931 2422 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.498517 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.473366 2261 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.482513 2429 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.482398 2446 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.464051 2334 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.482913 1742 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:59.462682 2424 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.509343 2261 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.538282 2409 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:59.539080 2404 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:15:59.585283 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 104719us
I20260619 21:15:59.585605 1859 tablet_bootstrap.cc:492] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:15:59.591658 1743 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:15:59.593184 1782 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:15:59.576651 2429 raft_consensus.cc:3037] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:15:59.483067 2326 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:59.591467 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 97933.4us
I20260619 21:15:59.482461 2435 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.589952 1740 raft_consensus.cc:2354] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:15:59.596148 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.598867 1667 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
W20260619 21:15:59.600314 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 110816us
W20260619 21:15:59.599514 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 119906us
I20260619 21:15:59.590157 2261 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.614791 2129 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.617040 2451 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.617717 2451 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:59.622012 2447 log.cc:924] Time spent T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.117s user 0.000s sys 0.000s
I20260619 21:15:59.595947 1666 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.595556 2409 raft_consensus.cc:686] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:15:59.658810 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:59.616745 2411 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:15:59.633049 2451 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.659139 2129 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.666679 2428 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:59.745731 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 148051us
I20260619 21:15:59.677881 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:59.765604 2417 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.766364 2430 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:15:59.746649 2419 consensus_meta.cc:287] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.597s user 0.007s sys 0.007s
I20260619 21:15:59.736431 2416 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.742951 2346 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.793316 1783 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:15:59.796597 1708 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.748992 2449 raft_consensus.cc:686] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:15:59.801246 1815 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.758028 2411 raft_consensus.cc:686] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:15:59.848220 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 105987us
W20260619 21:15:59.849059 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 52545.6us
W20260619 21:15:59.849273 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 56061.3us
I20260619 21:15:59.854251 1858 tablet_bootstrap.cc:654] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:59.854372 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.854317 1783 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.744102 2129 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.785328 1744 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 4
I20260619 21:15:59.875372 1743 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:15:59.880296 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.881237 1667 raft_consensus.cc:2354] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:15:59.792129 2423 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.890802 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:15:59.798004 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:15:59.846225 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.799945 2422 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:59.877135 2406 consensus_meta.cc:287] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.676s user 0.008s sys 0.006s
I20260619 21:15:59.898648 2448 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:15:59.884918 2230 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:15:59.902046 2457 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:15:59.875612 2334 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.892611 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 320) took 1039 ms. Trace:
I20260619 21:15:59.858673 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.858641 2324 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:15:59.943384 1859 tablet_bootstrap.cc:654] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:15:59.951646 1815 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:15:59.788722 2419 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.931710 1809 rpcz_store.cc:276] 0619 21:15:58.845160 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:58.897425 (+ 52265us) service_pool.cc:227] Handling call
0619 21:15:59.858209 (+960784us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540031f1d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:15:59.884439 (+ 26230us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":928003968}
I20260619 21:15:59.942358 1663 raft_consensus.cc:2379] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:15:59.966614 2449 consensus_queue.cc:237] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.968820 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.951186 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:15:59.969684 1663 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:15:59.943107 2324 raft_consensus.cc:686] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
W20260619 21:15:59.902839 2454 log.cc:924] Time spent T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Append to log took a long time: real 0.156s user 0.001s sys 0.000s
I20260619 21:15:59.788179 2409 consensus_queue.cc:237] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.967028 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:15:59.968973 2435 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.930588 1708 leader_election.cc:398] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:15:59.977103 1708 leader_election.cc:401] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:15:59.919752 2424 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:15:59.916438 2326 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.928463 2457 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:15:59.926710 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:15:59.920208 1662 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:15:59.979809 1741 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:15:59.934125 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:15:59.897984 2423 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.970021 1738 raft_consensus.cc:2354] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:15:59.983114 1735 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
W20260619 21:15:59.991425 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 61045.2us
I20260619 21:15:59.921276 2406 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:15:59.991123 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:00.011385 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 157132us
I20260619 21:16:00.003376 2419 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:15:59.986835 2265 consensus_meta.cc:287] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.625s user 0.007s sys 0.007s
I20260619 21:15:59.983842 2457 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.014593 1634 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:15:59.994603 1664 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:00.056636 2465 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:15:59.928730 2448 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.057899 2435 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:00.058509 2435 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:00.058988 2435 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:15:59.970400 2429 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.068529 1710 leader_election.cc:398] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4.
I20260619 21:16:00.068929 1710 leader_election.cc:401] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260619 21:16:00.059895 1666 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:00.060220 2448 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.023770 2265 raft_consensus.cc:2726] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:00.023197 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.073508 2423 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:15:59.976850 2411 consensus_queue.cc:237] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.014303 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.086658 1634 leader_election.cc:398] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:00.087064 1634 leader_election.cc:401] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:00.061342 2408 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.093356 1860 tablet_bootstrap.cc:492] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:00.094004 1860 ts_tablet_manager.cc:1397] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.942s user 0.013s sys 0.008s
I20260619 21:16:00.067767 2430 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.121155 1810 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:00.091329 2334 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:00.122642 2470 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 4
I20260619 21:16:00.125064 1662 raft_consensus.cc:2354] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:00.088351 2129 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.120610 2429 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.123605 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.142897 1782 leader_election.cc:398] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:00.060786 2324 consensus_queue.cc:237] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.147596 1782 leader_election.cc:401] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:00.149749 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.150758 1667 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:16:00.137971 2404 consensus_meta.cc:287] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.544s user 0.015s sys 0.000s
I20260619 21:16:00.141392 2427 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:00.157940 2404 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:00.098842 2408 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.156113 2430 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:00.158941 1738 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.158198 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.192262 1662 raft_consensus.cc:3037] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:00.187927 2406 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.190192 2408 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.192631 1860 raft_consensus.cc:348] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.205602 1860 raft_consensus.cc:374] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:00.206069 1860 raft_consensus.cc:729] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
W20260619 21:16:00.215524 2446 consensus_meta.cc:287] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.624s user 0.008s sys 0.007s
I20260619 21:16:00.206943 1860 consensus_queue.cc:260] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.216307 2446 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.236317 1817 raft_consensus.cc:2354] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:00.236968 2429 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.237802 2429 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.247396 1668 consensus_meta.cc:287] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.677s user 0.009s sys 0.006s
I20260619 21:16:00.247880 2446 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.247982 1668 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:00.236320 1818 raft_consensus.cc:2379] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:00.249879 2429 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.249992 1782 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:00.253688 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.255538 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.269074 1667 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:00.178052 2427 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.253675 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.281548 1860 ts_tablet_manager.cc:1428] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.187s user 0.008s sys 0.002s
I20260619 21:16:00.301769 1708 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:00.307780 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.310187 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.312779 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.311059 2425 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:00.308764 1635 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:00.308665 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.327683 2451 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:00.328282 2451 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:00.328742 2451 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:00.329897 1708 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:00.270581 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.320384 1667 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:16:00.325240 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:00.310559 1783 leader_election.cc:398] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:00.339617 1783 leader_election.cc:401] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260619 21:16:00.273648 2415 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.272221 2404 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.272369 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 316) took 1065 ms. Trace:
W20260619 21:16:00.308076 2341 consensus_meta.cc:287] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.804s user 0.006s sys 0.007s
I20260619 21:16:00.281749 2446 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
W20260619 21:16:00.333258 2421 consensus_meta.cc:287] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.937s user 0.011s sys 0.006s
I20260619 21:16:00.349316 2425 raft_consensus.cc:686] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:00.360754 2425 consensus_queue.cc:237] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.334178 2453 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.300139 2411 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:00.333693 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.348373 2474 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.374446 2427 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.380224 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 268) took 1394 ms. Trace:
I20260619 21:16:00.381296 2406 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.382000 2406 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.367856 2446 raft_consensus.cc:686] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:00.384003 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.389987 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:00.390046 2427 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.353529 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.354898 1744 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:00.392256 1736 raft_consensus.cc:2354] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:00.393985 2427 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.404964 2412 raft_consensus.cc:2726] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:00.393195 2406 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.402100 2263 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.402055 2436 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.350940 2404 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.416420 1667 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:00.350721 2415 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.354723 2341 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.411890 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 301) took 1025 ms. Trace:
I20260619 21:16:00.420765 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.363703 2421 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.422317 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.384588 2446 consensus_queue.cc:237] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.386003 2453 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.423622 2263 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.429502 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 337) took 1230 ms. Trace:
I20260619 21:16:00.434336 1664 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:00.436617 1634 leader_election.cc:398] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:00.442358 1708 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:00.442883 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.450887 1859 tablet_bootstrap.cc:492] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:00.475419 1634 leader_election.cc:401] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260619 21:16:00.457927 1710 leader_election.cc:341] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:00.476100 1859 ts_tablet_manager.cc:1397] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.906s user 0.019s sys 0.104s
I20260619 21:16:00.461555 1665 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:00.465175 2408 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.456326 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.480233 1663 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:00.480446 1665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 278) took 1267 ms. Trace:
I20260619 21:16:00.480019 2408 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.453920 1744 rpcz_store.cc:276] 0619 21:15:59.388130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.389897 (+ 1767us) service_pool.cc:227] Handling call
0619 21:15:59.664083 (+274186us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:00.405845 (+741762us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4654336}
I20260619 21:16:00.482215 2406 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:00.482751 2450 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:00.482988 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.484460 2408 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.450035 2415 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.453814 1810 raft_consensus.cc:2354] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4.
I20260619 21:16:00.452437 2341 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.490434 2406 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.491452 2406 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.481474 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 45099.5us
I20260619 21:16:00.467811 2470 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4.
I20260619 21:16:00.483523 2450 raft_consensus.cc:686] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:00.340718 1813 rpcz_store.cc:276] 0619 21:15:59.200711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.203165 (+ 2454us) service_pool.cc:227] Handling call
0619 21:16:00.155635 (+952470us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:00.172024 (+ 16389us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:00.195946 (+ 23922us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:00.205006 (+ 9060us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:00.219700 (+ 14694us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:00.226082 (+ 6382us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:00.244538 (+ 18456us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:00.266503 (+ 21965us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7361408}
I20260619 21:16:00.442778 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.464641 2453 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.490535 2427 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.372637 2411 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.486357 1811 raft_consensus.cc:2379] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:00.514868 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.515304 2406 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.515893 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.516175 2406 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.517562 2418 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.517115 2406 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.501191 2415 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.516613 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.503364 2458 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.518654 2415 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.519742 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.519479 2458 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.452816 2404 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.468143 1814 rpcz_store.cc:276] 0619 21:15:59.198882 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.201017 (+ 2135us) service_pool.cc:227] Handling call
0619 21:16:00.351309 (+1150292us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:00.429346 (+ 78037us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":56042752}
I20260619 21:16:00.500437 2408 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.454475 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.446230 2436 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.509268 1668 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.537870 2453 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.538653 2453 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.528256 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.541811 1814 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.550274 2458 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.553968 1635 leader_election.cc:398] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
W20260619 21:16:00.510416 2480 log.cc:924] Time spent T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.097s user 0.001s sys 0.000s
I20260619 21:16:00.509958 2450 consensus_queue.cc:237] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.517917 1710 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:00.579572 1710 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:00.518214 2418 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.449702 2261 consensus_meta.cc:287] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.819s user 0.012s sys 0.002s
I20260619 21:16:00.504508 1708 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:00.467856 2263 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.460538 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.526811 2408 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.473559 2421 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.528090 1812 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:00.481010 1665 rpcz_store.cc:276] 0619 21:15:59.213023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.213161 (+ 138us) service_pool.cc:227] Handling call
0619 21:16:00.480073 (+1266912us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400330254. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:00.480307 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2481128320}
I20260619 21:16:00.530849 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.598814 2464 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:00.599431 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.599740 2464 raft_consensus.cc:686] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:00.599953 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.531527 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.531898 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.513293 2411 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.561952 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.554539 2436 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:00.563396 2435 consensus_meta.cc:287] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.504s user 0.012s sys 0.004s
I20260619 21:16:00.564757 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.566452 2453 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.570297 2415 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.571739 1662 raft_consensus.cc:2454] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
W20260619 21:16:00.572471 2457 consensus_meta.cc:287] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.511s user 0.009s sys 0.006s
I20260619 21:16:00.548282 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:00.580757 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 62884.9us
W20260619 21:16:00.587844 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 83433us
I20260619 21:16:00.533768 1811 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 2
I20260619 21:16:00.591802 1668 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:00.600679 1665 raft_consensus.cc:2354] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2.
I20260619 21:16:00.607297 1782 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:00.585829 2458 raft_consensus.cc:2726] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:00.619511 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.637625 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.638353 1662 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:00.638726 1665 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:00.639182 2474 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:00.629541 2436 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.602941 1664 raft_consensus.cc:2354] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:00.592741 2418 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.606004 1735 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:00.620520 2457 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.602280 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.639891 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.682307 1782 leader_election.cc:398] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2.
I20260619 21:16:00.600898 1663 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:00.683419 1782 leader_election.cc:401] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260619 21:16:00.625380 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.622990 1708 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:00.683442 1858 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.536s user 0.117s sys 0.129s
I20260619 21:16:00.587085 2261 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.686281 1858 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.539s user 0.117s sys 0.130s
W20260619 21:16:00.686532 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 63668.8us
I20260619 21:16:00.640633 2411 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.688494 1782 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:00.689167 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.689596 1782 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:00.690863 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.641896 2421 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.641243 2404 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.660683 2263 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.652738 2408 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.697314 1708 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:00.699609 1783 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:00.700954 1708 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:00.680336 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:00.622303 2435 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.678656 2418 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.727941 1669 raft_consensus.cc:2354] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:00.730580 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.731843 1708 leader_election.cc:398] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:00.732259 1708 leader_election.cc:401] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:00.695755 2458 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:00.698788 2341 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:00.675637 2436 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.701026 2404 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.742259 2452 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.742959 2452 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.743647 2406 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.744720 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:00.745047 2419 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:00.745602 1736 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.745738 2419 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.746394 2419 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:00.695981 1817 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:00.750262 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.751802 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.751941 1859 raft_consensus.cc:348] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.753099 2341 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.753662 2341 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:00.754273 2475 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:00.757488 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.752840 1859 raft_consensus.cc:374] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:00.759016 1859 raft_consensus.cc:729] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:00.759049 1708 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:00.702495 2267 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:00.723172 2407 raft_consensus.cc:2781] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:00.768882 2261 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.738113 1809 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:00.601434 2464 consensus_queue.cc:237] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.380863 1661 rpcz_store.cc:276] 0619 21:15:58.985714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:58.989542 (+ 3828us) service_pool.cc:227] Handling call
0619 21:15:58.996480 (+ 6938us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:15:59.595559 (+599079us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:15:59.601047 (+ 5488us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:15:59.601741 (+ 694us) log.cc:842] Serialized 26 byte log entry
0619 21:15:59.763821 (+162080us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:15:59.784999 (+ 21178us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:00.379545 (+594546us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b540036e554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:00.379625 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:00.379703 (+ 78us) raft_consensus.cc:1671] finished
0619 21:16:00.379732 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:00.380081 (+ 349us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20433,"raft.queue_time_us":25860,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":12413,"spinlock_wait_cycles":1153811840,"thread_start_us":141779,"threads_started":1,"wal-append.queue_time_us":243646}
I20260619 21:16:00.775477 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.776566 1818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 314) took 1576 ms. Trace:
W20260619 21:16:00.777690 1666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 327) took 1619 ms (client timeout 2128 ms). Trace:
I20260619 21:16:00.723069 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:00.780984 1708 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:00.736860 1665 raft_consensus.cc:3037] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:00.728924 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.789917 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.810333 1817 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:00.759943 1859 consensus_queue.cc:260] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.734098 2474 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.738912 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.700846 2421 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.705123 2263 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.785842 1660 raft_consensus.cc:2354] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:00.757447 2456 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.817674 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.798558 1818 rpcz_store.cc:276] 0619 21:15:59.199987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.200641 (+ 654us) service_pool.cc:227] Handling call
0619 21:15:59.794994 (+594353us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:00.776257 (+981263us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400229054. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:00.776476 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":821430912}
I20260619 21:16:00.823851 1744 raft_consensus.cc:1261] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:00.826771 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 331) took 1370 ms. Trace:
I20260619 21:16:00.827241 1669 rpcz_store.cc:276] 0619 21:15:59.455951 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.494223 (+ 38272us) service_pool.cc:227] Handling call
0619 21:16:00.826558 (+1332335us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540036d154. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:00.826705 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2807390080}
I20260619 21:16:00.813418 2422 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:00.795953 2261 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:00.801287 1738 consensus_meta.cc:287] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.578s user 0.015s sys 0.000s
I20260619 21:16:00.853757 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 343) took 1249 ms. Trace:
I20260619 21:16:00.814384 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:00.789230 2415 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:00.789045 2452 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:00.734506 2418 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.788197 2468 log.cc:924] Time spent T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260619 21:16:00.872243 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:00.803469 1666 rpcz_store.cc:269] 0619 21:15:59.158711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.162650 (+ 3939us) service_pool.cc:227] Handling call
0619 21:16:00.777438 (+1614788us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54000aa354. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:00.777603 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":989391616}
I20260619 21:16:00.786928 2457 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:00.823694 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.779294 2458 raft_consensus.cc:686] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:00.812906 1710 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:00.795778 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.796383 2267 raft_consensus.cc:686] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:00.828678 2422 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:00.854321 1815 rpcz_store.cc:276] 0619 21:15:59.604211 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.605093 (+ 882us) service_pool.cc:227] Handling call
0619 21:16:00.853535 (+1248442us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400279554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:00.853683 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":325420288}
I20260619 21:16:00.863528 2463 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:00.856204 1738 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:00.864097 2415 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:16:00.922868 2444 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.308s user 0.044s sys 0.054s
I20260619 21:16:00.914618 1860 tablet_bootstrap.cc:492] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:00.788036 2451 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.936142 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 123328us
I20260619 21:16:00.787256 2436 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.000016 1710 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:00.913376 1635 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:00.993333 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:00.817749 2474 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.935115 2476 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.524s user 0.034s sys 0.057s
I20260619 21:16:00.835729 2456 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:01.001701 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 109871us
I20260619 21:16:00.931555 2452 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:00.865473 2261 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:00.993681 2444 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.414s user 0.044s sys 0.054s
I20260619 21:16:01.014124 1667 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:00.988566 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:00.993213 2415 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:01.032972 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 105768us
W20260619 21:16:01.006083 2476 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.595s user 0.034s sys 0.057s
I20260619 21:16:00.822849 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:00.995774 2417 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:01.037196 1708 leader_election.cc:398] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:01.015476 1859 ts_tablet_manager.cc:1428] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.539s user 0.018s sys 0.018s
I20260619 21:16:01.040549 1708 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:01.007567 2420 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:01.041790 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.005832 2425 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:16:01.040280 1859 ts_tablet_manager.cc:1460] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Tablet startup took 563ms
I20260619 21:16:01.003209 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 347) took 1022 ms. Trace:
W20260619 21:16:01.070309 1859 ts_tablet_manager.cc:1462] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Trace:
0619 21:15:42.711292 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:43.101062 (+389770us) service_pool.cc:227] Handling call
0619 21:15:43.122306 (+ 21244us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a6cd21 00007f71a3a4a1e3 00007f71a39fb869 00007f719cffdc44 00007f719cffdbd2 00007f719cffdb01
0619 21:15:43.153016 (+ 30710us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:15:43.153127 (+ 111us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:15:43.255000 (+101873us) tablet_metadata.cc:641] Metadata flushed
0619 21:15:43.275517 (+ 20517us) tablet_replica.cc:205] Creating consensus instance
0619 21:15:43.275973 (+ 456us) inbound_call.cc:173] Queueing success response
0619 21:15:59.539624 (+16263651us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:16:00.476581 (+936957us) ts_tablet_manager.cc:1429] Starting tablet replica
0619 21:16:00.749607 (+273026us) tablet_replica.cc:259] Starting instrumentation
0619 21:16:00.750283 (+ 676us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6421,"spinlock_wait_cycles":9029888,"tablet-open.queue_time_us":16230048}
I20260619 21:16:01.046444 1858 tablet_bootstrap.cc:492] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:01.047915 2263 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.051249 2455 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:01.052543 1782 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.016695 2452 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.053632 1664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 343) took 1170 ms. Trace:
I20260619 21:16:01.078763 2455 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.080332 2452 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.050113 1812 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
W20260619 21:16:01.054001 2484 log.cc:924] Time spent T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.204s user 0.001s sys 0.000s
I20260619 21:16:01.054097 2458 consensus_queue.cc:237] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.045239 2435 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.058570 2422 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.059063 2267 consensus_queue.cc:237] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.062441 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 290) took 1185 ms. Trace:
W20260619 21:16:01.064440 1814 consensus_meta.cc:287] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.522s user 0.035s sys 0.023s
I20260619 21:16:00.993889 2463 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.065510 2404 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.071643 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.101483 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 361) took 1005 ms. Trace:
I20260619 21:16:01.072054 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.074318 1858 ts_tablet_manager.cc:1397] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 1.908s user 0.142s sys 0.144s
I20260619 21:16:01.089222 1634 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.072767 1810 rpcz_store.cc:276] 0619 21:15:59.982111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.308306 (+326195us) service_pool.cc:227] Handling call
0619 21:16:00.987775 (+679469us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54000f8cd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:00.999990 (+ 12215us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":290384768}
I20260619 21:16:01.052354 1811 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:01.075040 1859 tablet_bootstrap.cc:492] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:01.074769 2425 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:01.071681 2444 consensus_queue.cc:1035] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260619 21:16:01.038134 2436 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.050348 2418 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.041738 2417 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:01.062469 2261 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.055527 1635 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.043566 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 338) took 1250 ms. Trace:
I20260619 21:16:01.086841 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:01.044601 1662 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:01.120160 1813 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:01.120236 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.121063 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.069136 2420 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.089460 1666 rpcz_store.cc:276] 0619 21:15:59.877280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.993886 (+1116606us) service_pool.cc:227] Handling call
0619 21:16:01.019882 (+ 25996us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.044785 (+ 24903us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.062101 (+ 17316us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.062161 (+ 60us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.062196 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.062267 (+ 71us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.062380 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.122437 1710 leader_election.cc:398] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4.
I20260619 21:16:01.092293 1809 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:01.092237 1736 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:01.122727 1710 leader_election.cc:401] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260619 21:16:01.092540 2463 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.093140 2455 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.093686 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.093788 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.093935 1814 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:01.095506 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.096195 2411 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.079712 1664 rpcz_store.cc:276] 0619 21:15:59.883335 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.052653 (+1169318us) service_pool.cc:227] Handling call
0619 21:16:01.053091 (+ 438us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.053272 (+ 181us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.053302 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.053347 (+ 45us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.053384 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.053443 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.053562 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.101840 1817 rpcz_store.cc:276] 0619 21:16:00.095752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.071370 (+975618us) service_pool.cc:227] Handling call
0619 21:16:01.101419 (+ 30049us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.149739 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.105976 2412 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2.
I20260619 21:16:01.106940 2474 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.111137 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:01.133011 2349 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:01.156244 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 346) took 1205 ms. Trace:
I20260619 21:16:01.086061 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 350) took 1099 ms. Trace:
I20260619 21:16:01.122292 2404 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.114944 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.115712 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 306) took 1328 ms. Trace:
I20260619 21:16:01.115969 1668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 340) took 1299 ms. Trace:
I20260619 21:16:01.116156 2425 raft_consensus.cc:686] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:01.116523 1811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 357) took 1096 ms. Trace:
I20260619 21:16:01.116386 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.118173 2417 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:01.118095 1735 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:01.102344 2263 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:01.118955 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.117631 1661 rpcz_store.cc:276] 0619 21:15:59.793589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.809926 (+1016337us) service_pool.cc:227] Handling call
0619 21:16:00.929732 (+119806us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:00.995206 (+ 65474us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:00.996482 (+ 1276us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.004077 (+ 7595us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.007402 (+ 3325us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.014449 (+ 7047us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.040299 (+ 25850us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.119923 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.158056 1812 rpcz_store.cc:276] 0619 21:15:59.986178 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.441295 (+455117us) service_pool.cc:227] Handling call
0619 21:16:01.084824 (+643529us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.123512 1667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 348) took 1113 ms. Trace:
I20260619 21:16:01.123755 1710 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:01.088084 2457 raft_consensus.cc:2726] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.108117 2451 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.121632 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:01.140270 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 84839.2us
I20260619 21:16:01.187569 1635 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.188527 1635 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.109479 2452 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.183692 1858 raft_consensus.cc:348] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.191249 1858 raft_consensus.cc:374] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:01.191638 1858 raft_consensus.cc:729] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:16:01.143934 2407 raft_consensus.cc:686] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:01.104017 1739 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
W20260619 21:16:01.195703 1739 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 303) took 1730 ms (client timeout 1765 ms). Trace:
I20260619 21:16:01.196085 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:01.196305 1739 rpcz_store.cc:269] 0619 21:15:59.465664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.504963 (+ 39299us) service_pool.cc:227] Handling call
0619 21:16:00.561736 (+1056773us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:01.195581 (+633845us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":52605312}
I20260619 21:16:01.197950 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.198722 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 291) took 1286 ms. Trace:
I20260619 21:16:01.198801 1810 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:01.199393 1666 rpcz_store.cc:276] 0619 21:15:59.912615 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.150767 (+1238152us) service_pool.cc:227] Handling call
0619 21:16:01.197866 (+ 47099us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.198139 (+ 273us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.198191 (+ 52us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.198302 (+ 111us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.198352 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.198453 (+ 101us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.198636 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.200201 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.121766 1662 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:01.201352 1815 raft_consensus.cc:2354] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:01.179684 2404 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.133435 2420 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:01.203806 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 359) took 1118 ms. Trace:
I20260619 21:16:01.153653 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.204778 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.205893 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.206372 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:01.153909 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.207109 1740 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:01.207892 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:01.210774 2425 consensus_queue.cc:237] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.212163 1660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 337) took 1422 ms. Trace:
I20260619 21:16:01.156585 1669 rpcz_store.cc:276] 0619 21:15:59.950945 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.155449 (+1204504us) service_pool.cc:227] Handling call
0619 21:16:01.155764 (+ 315us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.155938 (+ 174us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.155963 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.156005 (+ 42us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.156037 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.156090 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.156189 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.133337 2430 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.103241 2419 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.158427 1634 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:01.113624 2421 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.161429 1664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 345) took 1244 ms. Trace:
I20260619 21:16:01.161765 1663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 292) took 1177 ms. Trace:
I20260619 21:16:01.168340 1860 tablet_bootstrap.cc:654] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:01.169665 2490 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.155617 2230 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.146152 2418 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.121078 2436 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.229830 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 354) took 1241 ms. Trace:
I20260619 21:16:01.151854 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 356) took 1139 ms. Trace:
I20260619 21:16:01.204185 2404 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.183300 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:01.183314 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 59619.3us
I20260619 21:16:01.181756 1667 rpcz_store.cc:276] 0619 21:16:00.009956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.441821 (+431865us) service_pool.cc:227] Handling call
0619 21:16:01.123325 (+681504us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400150c54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:01.123456 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1239585408}
I20260619 21:16:01.193840 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.194960 2453 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.216060 1708 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:01.207127 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.204236 1815 rpcz_store.cc:276] 0619 21:16:00.085855 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.993016 (+907161us) service_pool.cc:227] Handling call
0619 21:16:01.203569 (+210553us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54000f87d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:01.203733 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":458670464}
I20260619 21:16:01.192458 1858 consensus_queue.cc:260] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.218016 1783 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:01.220556 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 62225us
I20260619 21:16:01.179195 2326 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4.
I20260619 21:16:01.241091 2404 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.241345 2418 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:01.242403 2436 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.242000 2418 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.243072 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.244352 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.178597 1811 rpcz_store.cc:276] 0619 21:16:00.013736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.516229 (+502493us) service_pool.cc:227] Handling call
0619 21:16:01.110131 (+593902us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.221527 2430 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.241887 2404 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.222959 1663 rpcz_store.cc:276] 0619 21:15:59.984851 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.160634 (+1175783us) service_pool.cc:227] Handling call
0619 21:16:01.161085 (+ 451us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.161319 (+ 234us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.161361 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.161422 (+ 61us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.161475 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.161548 (+ 73us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.161688 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.185178 2263 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.226835 2419 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.226635 2341 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.224105 2455 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:01.255105 2455 raft_consensus.cc:686] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:01.234910 1814 rpcz_store.cc:276] 0619 21:16:00.011011 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.521598 (+510587us) service_pool.cc:227] Handling call
0619 21:16:01.150354 (+628756us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.183575 2334 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.228118 2490 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.183193 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.185868 2495 raft_consensus.cc:2726] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.231058 1743 raft_consensus.cc:1226] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:01.212034 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.176100 1742 rpcz_store.cc:276] 0619 21:15:59.787122 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.787257 (+ 135us) service_pool.cc:227] Handling call
0619 21:16:00.864717 (+1077460us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:00.937263 (+ 72546us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.020265 (+ 83002us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.036745 (+ 16480us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.041845 (+ 5100us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.045262 (+ 3417us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.115433 (+ 70171us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b54002d5ad4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.115487 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.115625 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":156255872}
I20260619 21:16:01.252403 2453 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:01.279560 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.178834 1668 rpcz_store.cc:276] 0619 21:15:59.816280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.819152 (+1002872us) service_pool.cc:227] Handling call
0619 21:16:00.822618 (+ 3466us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:00.925830 (+103212us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:00.929883 (+ 4053us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:00.933604 (+ 3721us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:00.989302 (+ 55698us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.115717 (+126415us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b5400332a54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.115767 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.115902 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":98904832}
W20260619 21:16:01.237278 1710 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:01.221973 2452 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.239410 1817 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:01.212656 1660 rpcz_store.cc:276] 0619 21:15:59.789354 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.515423 (+726069us) service_pool.cc:227] Handling call
0619 21:16:01.211858 (+696435us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54000a9e54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:01.212065 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":569385856}
I20260619 21:16:01.221513 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.222246 1635 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:01.291883 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 69724.8us
I20260619 21:16:01.292591 1635 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.234184 2230 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.256556 2455 consensus_queue.cc:237] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.256407 2263 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.301859 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.303125 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.304284 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.260677 1858 ts_tablet_manager.cc:1428] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.158s user 0.006s sys 0.002s
W20260619 21:16:01.245841 2493 log.cc:924] Time spent T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260619 21:16:01.283264 2453 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.249365 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 348) took 1264 ms. Trace:
I20260619 21:16:01.311220 1815 rpcz_store.cc:276] 0619 21:15:59.985454 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.246428 (+1260974us) service_pool.cc:227] Handling call
0619 21:16:01.246809 (+ 381us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.249067 (+ 2258us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.249097 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.249137 (+ 40us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.249170 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.249220 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.249312 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.312284 2453 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.312984 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 353) took 1324 ms. Trace:
I20260619 21:16:01.306784 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:01.313370 1815 rpcz_store.cc:276] 0619 21:15:59.988121 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.312186 (+1324065us) service_pool.cc:227] Handling call
0619 21:16:01.312493 (+ 307us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.312674 (+ 181us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.312703 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.312743 (+ 40us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.312772 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.312813 (+ 41us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.312906 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.314998 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 355) took 1317 ms. Trace:
I20260619 21:16:01.251964 1810 raft_consensus.cc:2379] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:01.249100 2474 raft_consensus.cc:2781] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:01.245564 2349 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:01.221659 1664 rpcz_store.cc:276] 0619 21:15:59.917193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.155702 (+1238509us) service_pool.cc:227] Handling call
0619 21:16:01.160801 (+ 5099us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.161194 (+ 393us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.161368 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.248832 2430 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.276862 1816 raft_consensus.cc:1226] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:01.233666 1813 rpcz_store.cc:276] 0619 21:15:59.988381 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.519300 (+530919us) service_pool.cc:227] Handling call
0619 21:16:01.229564 (+710264us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400260a54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:01.229743 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":275961216}
I20260619 21:16:01.336853 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 331) took 1310 ms. Trace:
I20260619 21:16:01.232343 1859 tablet_bootstrap.cc:654] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:01.337596 2490 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.244892 2418 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.287530 1710 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:01.338766 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 101517us
I20260619 21:16:01.340010 1710 leader_election.cc:398] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:01.287196 2421 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:01.247527 2415 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.263964 2334 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.294729 1667 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:01.297118 2267 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.297518 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.281226 1736 raft_consensus.cc:2454] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:16:01.299306 2456 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.299886 2435 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.299839 1635 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:01.350910 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 51154.8us
I20260619 21:16:01.351630 1635 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.270552 2407 consensus_queue.cc:237] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.306978 1812 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:01.315300 1815 rpcz_store.cc:276] 0619 21:15:59.997456 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.314275 (+1316819us) service_pool.cc:227] Handling call
0619 21:16:01.314552 (+ 277us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.314714 (+ 162us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.314742 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.314779 (+ 37us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.314807 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.314855 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.314952 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.315740 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 352) took 1307 ms. Trace:
I20260619 21:16:01.315284 1663 raft_consensus.cc:1261] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.291599 2404 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.332963 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.335475 1858 tablet_bootstrap.cc:492] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:01.335304 2474 raft_consensus.cc:686] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:01.295898 1708 leader_election.cc:398] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:01.366834 1708 leader_election.cc:401] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:01.344406 1664 raft_consensus.cc:1261] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260619 21:16:01.345389 2496 log.cc:924] Time spent T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260619 21:16:01.345835 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:01.337230 1813 rpcz_store.cc:276] 0619 21:16:00.026887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.335972 (+1309085us) service_pool.cc:227] Handling call
0619 21:16:01.336304 (+ 332us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.336492 (+ 188us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.336521 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.336563 (+ 42us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.336601 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.336654 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.336790 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.300503 2491 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.345705 2425 raft_consensus.cc:2726] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:01.334030 2349 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:01.352787 1635 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.379540 1635 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.380468 1635 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.381618 1635 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.358155 1817 rpcz_store.cc:276] 0619 21:15:59.987826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.294565 (+1306739us) service_pool.cc:227] Handling call
0619 21:16:01.294877 (+ 312us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.352089 2418 raft_consensus.cc:2726] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.346944 2421 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.346683 2415 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.306149 2417 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.353428 2263 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:01.349576 2430 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:01.368482 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 72695.6us
I20260619 21:16:01.354889 1744 raft_consensus.cc:1226] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
W20260619 21:16:01.383282 1710 consensus_peers.cc:487] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:16:01.384150 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 333) took 1185 ms. Trace:
I20260619 21:16:01.383260 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.395380 1817 rpcz_store.cc:276] 0619 21:16:00.198860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.383396 (+1184536us) service_pool.cc:227] Handling call
0619 21:16:01.383680 (+ 284us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.383839 (+ 159us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.383866 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.383918 (+ 52us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.383946 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.383996 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.384093 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.396185 1635 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.351476 2267 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.363332 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.397914 1634 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:01.376614 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.382633 2451 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.381153 2349 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:01.402814 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.398800 1708 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:01.381335 2452 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260619 21:16:01.394783 1665 consensus_meta.cc:287] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.567s user 0.013s sys 0.000s
I20260619 21:16:01.384569 2435 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.385077 2491 raft_consensus.cc:2726] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.408612 1741 raft_consensus.cc:2396] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:01.386152 2263 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.400130 1818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 358) took 1382 ms. Trace:
I20260619 21:16:01.401634 2453 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:01.399734 2418 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:01.402868 2406 raft_consensus.cc:2781] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:16:01.401611 2404 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.404763 2334 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.403221 2326 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:01.456100 2326 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.456449 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.458133 2326 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.441541 1782 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.458811 1660 raft_consensus.cc:1261] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.459837 2490 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:01.460146 1782 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.460404 1660 raft_consensus.cc:1261] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.462131 2417 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.464979 1663 raft_consensus.cc:1261] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.424768 1818 rpcz_store.cc:276] 0619 21:16:00.015484 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.332918 (+1317434us) service_pool.cc:227] Handling call
0619 21:16:01.352094 (+ 19176us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.363084 (+ 10990us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.368759 (+ 5675us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.376915 (+ 8156us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.379503 (+ 2588us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.383992 (+ 4489us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.397300 (+ 13308us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.467033 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.468122 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 366) took 1289 ms. Trace:
I20260619 21:16:01.468492 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.444916 2267 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.448514 2230 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.410179 2461 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.451227 1563 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37734 (request call id 80) took 3613 ms. Trace:
I20260619 21:16:01.452495 1735 raft_consensus.cc:1261] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:01.442236 2419 consensus_queue.cc:1046] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260619 21:16:01.453799 1743 raft_consensus.cc:1261] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:01.420091 2263 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:01.430672 1737 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:01.460210 2490 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:01.469245 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:01.438824 2406 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:01.471855 1565 catalog_manager.cc:5495] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 1 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:16:01.434152 2418 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.473765 1811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 329) took 1513 ms. Trace:
I20260619 21:16:01.476763 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:01.548648 2460 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:01.549912 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.550441 2460 consensus_queue.cc:1035] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:01.551232 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.419001 2464 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.397686 2430 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.554105 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 351) took 1567 ms. Trace:
I20260619 21:16:01.554966 2430 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.407843 1665 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:01.555622 1665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 299) took 1097 ms. Trace:
I20260619 21:16:01.555959 1665 rpcz_store.cc:276] 0619 21:16:00.457773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.639825 (+182052us) service_pool.cc:227] Handling call
0619 21:16:01.555558 (+915733us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.556838 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.558037 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.494410 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:01.494827 2341 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.495262 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.495431 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.496420 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 355) took 1230 ms. Trace:
I20260619 21:16:01.524407 2490 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:01.573184 1669 rpcz_store.cc:276] 0619 21:16:00.266261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.216047 (+949786us) service_pool.cc:227] Handling call
0619 21:16:01.216329 (+ 282us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.345111 (+128782us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.345144 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.345193 (+ 49us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.345220 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.496194 (+150974us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54002e3454. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.496259 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.496361 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":295774592}
I20260619 21:16:01.498315 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.443393 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.492380 1563 rpcz_store.cc:276] 0619 21:15:57.838294 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:57.838457 (+ 163us) service_pool.cc:227] Handling call
0619 21:16:01.451158 (+3612701us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:15:59.273156 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:15:59.431240 (+158084us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:15:59.431314 (+ 74us) write_op.cc:435] Acquired schema lock
0619 21:15:59.431355 (+ 41us) tablet.cc:660] Decoding operations
0619 21:15:59.441571 (+ 10216us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:15:59.441686 (+ 115us) write_op.cc:641] Partition lock acquired for write op
0619 21:15:59.441751 (+ 65us) tablet.cc:683] Acquiring locks for 1 operations
0619 21:15:59.442029 (+ 278us) tablet.cc:699] Row locks acquired
0619 21:15:59.442063 (+ 34us) write_op.cc:260] PREPARE: finished
0619 21:15:59.442269 (+ 206us) write_op.cc:270] Start()
0619 21:15:59.442417 (+ 148us) write_op.cc:276] Timestamp: P: 1781903759442210 usec, L: 0
0619 21:15:59.442449 (+ 32us) op_driver.cc:331] REPLICATION: starting
0619 21:15:59.443419 (+ 970us) log.cc:842] Serialized 531 byte log entry
0619 21:16:00.410548 (+967129us) op_driver.cc:444] REPLICATION: finished
0619 21:16:01.066893 (+656345us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b1800015640. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb96638 00007f719bb99b11 00007f719bb99ac9 00007f719bb99a59 00007f719bb99a21 00007f719bb98d1d 00007f71a4dfa204 00007f71a4dfa039 00007f719bb78dc6
0619 21:16:01.066994 (+ 101us) write_op.cc:301] APPLY: starting
0619 21:16:01.067056 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0619 21:16:01.067132 (+ 76us) tablet.cc:1368] finished BulkCheckPresence
0619 21:16:01.067145 (+ 13us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:16:01.067339 (+ 194us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:16:01.067466 (+ 127us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0619 21:16:01.067493 (+ 27us) write_op.cc:312] APPLY: finished
0619 21:16:01.077236 (+ 9743us) log.cc:842] Serialized 22 byte log entry
0619 21:16:01.077438 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:01.077538 (+ 100us) write_op.cc:454] Released schema lock
0619 21:16:01.077585 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0619 21:16:01.077862 (+ 277us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:01.077894 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10510,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":637562,"apply.run_cpu_time_us":1923,"apply.run_wall_time_us":11107,"num_ops":1,"prepare.queue_time_us":364191,"prepare.run_cpu_time_us":37316,"prepare.run_wall_time_us":326349,"raft.queue_time_us":386253,"raft.run_cpu_time_us":92954,"raft.run_wall_time_us":631609,"replication_time_us":967952,"spinlock_wait_cycles":39308928,"thread_start_us":1176473,"threads_started":4,"wal-append.queue_time_us":225346}]]}
I20260619 21:16:01.506330 2461 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.507105 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 295) took 1347 ms. Trace:
I20260619 21:16:01.588565 1859 tablet_bootstrap.cc:492] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:01.587579 1666 rpcz_store.cc:276] 0619 21:16:00.159407 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.200914 (+1041507us) service_pool.cc:227] Handling call
0619 21:16:01.506332 (+305418us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:01.506561 (+ 229us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.506745 (+ 184us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.506771 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.506816 (+ 45us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.506846 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.506903 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.507017 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7853312}
I20260619 21:16:01.601878 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.427201 2453 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:01.611711 2430 consensus_queue.cc:1046] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:01.612234 2453 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:01.612766 2430 raft_consensus.cc:3032] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:01.613171 2430 raft_consensus.cc:729] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:01.613970 2430 consensus_queue.cc:260] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.473096 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 332) took 1275 ms. Trace:
I20260619 21:16:01.615005 1813 rpcz_store.cc:276] 0619 21:16:00.197929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.383232 (+1185303us) service_pool.cc:227] Handling call
0619 21:16:01.407692 (+ 24460us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.425760 (+ 18068us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.428203 (+ 2443us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.439971 (+ 11768us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.442990 (+ 3019us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.472850 (+ 29860us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.473019 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.616503 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.578635 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.617843 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.618260 2424 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:01.535378 2436 consensus_queue.cc:1035] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
I20260619 21:16:01.619026 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.620134 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.468642 1812 rpcz_store.cc:276] 0619 21:16:00.178959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.377119 (+1198160us) service_pool.cc:227] Handling call
0619 21:16:01.467165 (+ 90046us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:01.467422 (+ 257us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.467670 (+ 248us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.467721 (+ 51us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.467777 (+ 56us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.467833 (+ 56us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.467910 (+ 77us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.468042 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3770112}
I20260619 21:16:01.534852 2406 raft_consensus.cc:3037] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:01.621227 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.622404 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.623487 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.624012 1812 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:01.532265 1635 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.420327 2465 consensus_queue.cc:1035] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.099s
I20260619 21:16:01.617868 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.541379 1811 rpcz_store.cc:276] 0619 21:15:59.960791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.246191 (+1285400us) service_pool.cc:227] Handling call
0619 21:16:01.246545 (+ 354us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.254164 (+ 7619us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.254239 (+ 75us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.254302 (+ 63us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.254367 (+ 65us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.473447 (+219080us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b5400278654. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.473521 (+ 74us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.473693 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8678656}
I20260619 21:16:01.561848 1708 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:01.626264 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:01.626576 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.627058 1666 raft_consensus.cc:2379] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:01.491662 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 364) took 1302 ms. Trace:
I20260619 21:16:01.496644 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.455024 2421 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.542613 2418 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:01.583792 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.479329 2495 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260619 21:16:01.498510 2404 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.586146 2420 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.508685 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 365) took 1343 ms. Trace:
I20260619 21:16:01.384729 2474 consensus_queue.cc:237] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.554495 1814 rpcz_store.cc:276] 0619 21:15:59.986542 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.256112 (+1269570us) service_pool.cc:227] Handling call
0619 21:16:01.355964 (+ 99852us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.365058 (+ 9094us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.365106 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.365155 (+ 49us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.365205 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.365269 (+ 64us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.553916 (+188647us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b540023d2d0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabda09 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:01.554036 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":224226176}
I20260619 21:16:01.523916 2230 consensus_queue.cc:1035] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260619 21:16:01.559077 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.657656 1664 raft_consensus.cc:1261] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:01.659135 2404 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.661509 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 368) took 1215 ms. Trace:
I20260619 21:16:01.661648 2263 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.662227 1814 rpcz_store.cc:276] 0619 21:16:00.446048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.660352 (+1214304us) service_pool.cc:227] Handling call
0619 21:16:01.660703 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.660907 (+ 204us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.660961 (+ 54us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.661050 (+ 89us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.661112 (+ 62us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.661245 (+ 133us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.661424 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.664546 2263 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.638412 2412 consensus_queue.cc:1035] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260619 21:16:01.580925 2457 consensus_queue.cc:1046] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260619 21:16:01.666320 2457 raft_consensus.cc:3032] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:01.643042 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.666839 2457 raft_consensus.cc:729] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:01.670161 1809 rpcz_store.cc:276] 0619 21:16:00.165481 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.369746 (+1204265us) service_pool.cc:227] Handling call
0619 21:16:01.507999 (+138253us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:01.508157 (+ 158us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.508351 (+ 194us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.508377 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.508423 (+ 46us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.508455 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.508513 (+ 58us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.508623 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3544448}
I20260619 21:16:01.672232 2430 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:01.673090 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.591358 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 294) took 1502 ms. Trace:
I20260619 21:16:01.675163 1661 rpcz_store.cc:276] 0619 21:16:00.089012 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.189890 (+1100878us) service_pool.cc:227] Handling call
0619 21:16:01.238346 (+ 48456us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.316470 (+ 78124us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.324595 (+ 8125us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.331245 (+ 6650us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.334785 (+ 3540us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.578790 (+244005us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b540034fb54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.580632 (+ 1842us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.591282 (+ 10650us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":130335616}
I20260619 21:16:01.606487 1859 ts_tablet_manager.cc:1397] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.532s user 0.013s sys 0.017s
I20260619 21:16:01.599924 2461 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:16:01.576148 2341 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.609328 2415 raft_consensus.cc:3032] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:01.611126 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:16:01.628603 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 96468us
I20260619 21:16:01.687316 1782 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.692203 1635 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:01.638247 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 76487.8us
I20260619 21:16:01.579852 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.578298 1740 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:01.607959 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.668067 2457 consensus_queue.cc:260] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.607223 1816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 338) took 2383 ms. Trace:
I20260619 21:16:01.569653 2417 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:01.683488 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 337) took 1376 ms. Trace:
I20260619 21:16:01.718743 1815 rpcz_store.cc:276] 0619 21:16:00.307572 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.404035 (+1096463us) service_pool.cc:227] Handling call
0619 21:16:01.423300 (+ 19265us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.423579 (+ 279us) raft_consensus.cc:1520] Early marking committed up to index 1
0619 21:16:01.572902 (+149323us) log.cc:842] Serialized 14 byte log entry
0619 21:16:01.683142 (+110240us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.683209 (+ 67us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.683252 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.683321 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.683433 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":227998,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":6986,"spinlock_wait_cycles":50365696,"thread_start_us":228832,"threads_started":2}
I20260619 21:16:01.636246 1810 rpcz_store.cc:276] 0619 21:16:00.139367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.363945 (+1224578us) service_pool.cc:227] Handling call
0619 21:16:01.382703 (+ 18758us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.401742 (+ 19039us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.403231 (+ 1489us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.406980 (+ 3749us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.415355 (+ 8375us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.426008 (+ 10653us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.441321 (+ 15313us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.697803 1858 tablet_bootstrap.cc:654] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:01.654747 1669 raft_consensus.cc:1261] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260619 21:16:01.661679 2498 log.cc:924] Time spent T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.219s user 0.000s sys 0.001s
I20260619 21:16:01.715304 1635 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.717093 1782 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.725481 2422 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:01.726508 2420 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.693356 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 339) took 1247 ms. Trace:
I20260619 21:16:01.699275 2341 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.693464 1818 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:01.695937 2263 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:01.684969 2456 raft_consensus.cc:2726] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.788034 1860 tablet_bootstrap.cc:492] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:01.788578 1860 ts_tablet_manager.cc:1397] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.874s user 0.019s sys 0.030s
I20260619 21:16:01.730791 1662 raft_consensus.cc:1226] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:01.734503 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:01.726886 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 373) took 1048 ms. Trace:
I20260619 21:16:01.737632 1811 raft_consensus.cc:1261] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.749281 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:01.749966 2334 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:01.750576 2500 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.755764 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 334) took 1534 ms. Trace:
I20260619 21:16:01.786844 2421 raft_consensus.cc:2726] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.749313 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.641857 2465 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260619 21:16:01.767182 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 50224.9us
I20260619 21:16:01.810204 1782 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.810781 1668 raft_consensus.cc:1261] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:01.728515 1859 raft_consensus.cc:348] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.732595 2418 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.777077 2444 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.696319 2415 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:01.807569 2334 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.821452 2415 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.823235 2334 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:01.778311 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 63245.8us
I20260619 21:16:01.816220 2474 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:01.807631 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.826462 1665 raft_consensus.cc:1226] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:01.775730 1708 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:01.696766 2267 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:01.796188 2406 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.781875 1814 rpcz_store.cc:276] 0619 21:16:00.446418 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.663948 (+1217530us) service_pool.cc:227] Handling call
0619 21:16:01.664381 (+ 433us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.664616 (+ 235us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.664653 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.664723 (+ 70us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.664768 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.664859 (+ 91us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.684766 (+ 19907us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.798250 1809 rpcz_store.cc:276] 0619 21:16:00.678924 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.672029 (+993105us) service_pool.cc:227] Handling call
0619 21:16:01.672567 (+ 538us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.672855 (+ 288us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.672896 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.686584 (+ 13688us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:01.696701 (+ 10117us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.708517 (+ 11816us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.726794 (+ 18277us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.792508 1818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 372) took 1123 ms. Trace:
I20260619 21:16:01.766454 2417 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.805138 1817 rpcz_store.cc:276] 0619 21:16:00.222184 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.400979 (+1178795us) service_pool.cc:227] Handling call
0619 21:16:01.463172 (+ 62193us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:01.463370 (+ 198us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.599549 (+136179us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.599580 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:01.599618 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:01.599644 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.740883 (+141239us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540002fdd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.755520 (+ 14637us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:01.755689 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":318984192}
I20260619 21:16:01.744664 2453 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.732846 2230 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.833529 2267 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.816394 2419 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260619 21:16:01.835657 2453 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.838686 2406 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.760589 1816 rpcz_store.cc:276] 0619 21:15:59.199268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:59.201384 (+ 2116us) service_pool.cc:227] Handling call
0619 21:15:59.742467 (+541083us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:00.813211 (+1070744us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:00.822147 (+ 8936us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.335496 (+513349us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.566682 (+231186us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400115dd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.579241 (+ 12559us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1084201984}
I20260619 21:16:01.840588 1816 raft_consensus.cc:1261] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260619 21:16:01.842242 1809 raft_consensus.cc:1261] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.788897 2457 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:01.843703 2341 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.812657 1859 raft_consensus.cc:374] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:01.844532 1859 raft_consensus.cc:729] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:01.845309 1859 consensus_queue.cc:260] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.846987 2490 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.793433 1810 raft_consensus.cc:1261] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.792647 2491 raft_consensus.cc:2726] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.817341 1860 raft_consensus.cc:348] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.848528 1860 raft_consensus.cc:374] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:01.848876 1860 raft_consensus.cc:729] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:01.849653 1860 consensus_queue.cc:260] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.853557 1816 raft_consensus.cc:1226] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:01.791342 2263 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.830287 1818 rpcz_store.cc:276] 0619 21:16:00.666283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.668233 (+1001950us) service_pool.cc:227] Handling call
0619 21:16:01.668727 (+ 494us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.789741 (+121014us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.790030 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:01.857391 2417 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.860337 2465 consensus_queue.cc:1046] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:01.862334 2407 raft_consensus.cc:3032] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:01.862095 2465 consensus_queue.cc:1035] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:01.862835 2407 raft_consensus.cc:729] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:01.865654 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.866366 1812 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:01.866501 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:01.867583 2326 consensus_queue.cc:1035] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:01.820113 2418 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.812084 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:01.870702 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 58666.8us
W20260619 21:16:01.824571 1635 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:16:01.778645 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 163067us
I20260619 21:16:01.829196 1667 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:01.871415 1782 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.847941 1859 ts_tablet_manager.cc:1428] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.170s user 0.008s sys 0.000s
W20260619 21:16:01.871712 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 47188.5us
I20260619 21:16:01.872414 1635 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:01.828969 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 53367us
I20260619 21:16:01.874933 1860 ts_tablet_manager.cc:1428] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.086s user 0.008s sys 0.000s
I20260619 21:16:01.880578 1817 raft_consensus.cc:1261] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:01.881711 1635 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.823256 2415 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:01.884312 2230 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:01.885073 2456 raft_consensus.cc:2726] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.885000 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.887722 2455 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:01.888561 2455 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.893260 1782 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.895066 1708 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:01.896136 1783 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.898854 1783 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:01.894131 2495 consensus_queue.cc:1035] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260619 21:16:01.901374 2425 raft_consensus.cc:2726] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.894277 2500 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.903723 2456 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.889470 2349 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:01.890280 2341 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.905632 2263 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.906404 2263 raft_consensus.cc:2726] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.906814 1635 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.831581 2458 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.907212 2349 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.907996 2334 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:01.908186 2230 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.908396 1811 raft_consensus.cc:1261] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.908514 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.909502 1859 tablet_bootstrap.cc:492] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:01.836934 1662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 354) took 1580 ms. Trace:
I20260619 21:16:01.905333 2422 consensus_queue.cc:1035] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:01.864084 2407 consensus_queue.cc:260] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:01.914443 1635 leader_election.cc:341] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:01.914992 1635 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.918952 2425 consensus_queue.cc:1046] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260619 21:16:01.912461 1710 leader_election.cc:341] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:01.923120 1783 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.876281 2267 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.933209 1783 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.937160 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.946872 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.907169 2326 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260619 21:16:01.950865 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:01.941620 2456 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:01.952728 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:01.954739 1665 raft_consensus.cc:1261] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:01.911367 2455 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:01.910511 1662 rpcz_store.cc:276] 0619 21:16:00.256903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.202960 (+946057us) service_pool.cc:227] Handling call
0619 21:16:01.203401 (+ 441us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:01.795747 (+592346us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:01.836544 (+ 40797us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400332554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:01.836830 (+ 286us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1141155072}
I20260619 21:16:01.966818 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.969446 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.939878 2424 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:02.001392 2424 raft_consensus.cc:686] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:01.973574 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:01.941392 2419 consensus_queue.cc:1046] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260619 21:16:01.934561 2230 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:01.977695 2490 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:01.978654 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:01.979707 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:16:01.990453 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 44625.6us
I20260619 21:16:01.964774 2464 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260619 21:16:01.994356 2495 raft_consensus.cc:3032] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:01.996006 2407 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:01.986449 2465 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260619 21:16:01.964347 2349 consensus_queue.cc:1046] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260619 21:16:01.967008 1860 tablet_bootstrap.cc:492] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:02.026214 1635 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.033665 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:02.049597 1782 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.015305 2419 raft_consensus.cc:3032] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:02.059322 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.060549 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.003046 2424 consensus_queue.cc:237] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.064368 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:02.026405 2455 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:02.027931 2495 raft_consensus.cc:729] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:02.070705 1858 tablet_bootstrap.cc:492] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:02.080900 1858 ts_tablet_manager.cc:1397] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.747s user 0.012s sys 0.017s
I20260619 21:16:02.072571 1814 raft_consensus.cc:1261] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260619 21:16:02.074241 1635 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:02.085258 2261 consensus_queue.cc:1035] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:02.048375 2418 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:02.063599 2430 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:02.057976 2419 raft_consensus.cc:729] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:02.079299 2349 raft_consensus.cc:3032] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:02.079277 2334 raft_consensus.cc:2726] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:02.089888 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:16:02.129626 1710 consensus_peers.cc:487] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:16:02.092391 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:02.098073 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:02.110955 2418 raft_consensus.cc:686] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:02.114199 2349 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
W20260619 21:16:02.177870 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 93799.3us
W20260619 21:16:02.193910 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 134529us
I20260619 21:16:02.194722 1782 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.169198 2421 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:02.205284 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:02.199311 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:02.201066 2412 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:02.203896 2349 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.211448 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:02.214123 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:02.207376 1782 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.248883 2453 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:02.249781 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:02.249941 2453 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:02.255972 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 48665.4us
I20260619 21:16:02.246948 2500 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:02.157552 2495 consensus_queue.cc:260] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.182410 2419 consensus_queue.cc:260] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.257584 2500 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.352331 1782 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:02.325286 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 73023.5us
I20260619 21:16:02.354724 1782 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.245862 1738 raft_consensus.cc:3037] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:02.350123 2522 log.cc:924] Time spent T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.105s user 0.002s sys 0.000s
I20260619 21:16:02.353479 2457 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:02.361568 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.252467 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:02.350948 2453 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.293906 2430 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:02.340396 2455 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:02.387780 2455 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:02.388269 2513 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.281s user 0.000s sys 0.003s
I20260619 21:16:02.389149 1735 raft_consensus.cc:1261] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260619 21:16:02.389433 2518 log.cc:924] Time spent T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.191s user 0.000s sys 0.001s
I20260619 21:16:02.390004 2455 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.319926 2418 consensus_queue.cc:237] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.386709 1860 tablet_bootstrap.cc:654] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:02.262252 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:02.393817 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.394490 1814 raft_consensus.cc:2454] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:16:02.400481 1635 leader_election.cc:302] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:02.374249 2528 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260619 21:16:02.407562 1708 leader_election.cc:341] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:02.410452 1708 leader_election.cc:302] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:02.411823 1708 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:02.417964 2516 log.cc:924] Time spent T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.248s user 0.000s sys 0.001s
W20260619 21:16:02.425712 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 68524.4us
I20260619 21:16:02.421051 1858 raft_consensus.cc:348] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.431411 1858 raft_consensus.cc:374] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:02.431792 1858 raft_consensus.cc:729] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:16:02.437284 2536 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:02.432621 1858 consensus_queue.cc:260] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.413702 2419 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:02.412257 2534 raft_consensus.cc:2781] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260619 21:16:02.457862 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 411346us
W20260619 21:16:02.459185 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 77008.2us
I20260619 21:16:02.458894 1783 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.433336 2470 raft_consensus.cc:2726] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:02.461187 1783 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.462388 1858 ts_tablet_manager.cc:1428] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.381s user 0.008s sys 0.010s
I20260619 21:16:02.460059 1634 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.464386 1634 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.468189 1783 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.463066 2267 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:02.458341 2534 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:02.469828 2457 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:02.472355 2263 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:02.473546 1662 raft_consensus.cc:1261] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:02.474709 2455 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:02.488771 1858 tablet_bootstrap.cc:492] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:02.471944 2490 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:02.491621 1735 raft_consensus.cc:1261] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:02.492431 1738 raft_consensus.cc:2454] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:02.495872 2411 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:02.502451 2411 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.506471 1744 raft_consensus.cc:1226] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:02.530673 2411 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:02.537683 2500 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.539194 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:02.541239 1635 leader_election.cc:341] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:02.542567 2508 consensus_queue.cc:1035] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:02.486986 2534 raft_consensus.cc:3037] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:02.627686 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 130024us
I20260619 21:16:02.657763 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:02.659065 1743 raft_consensus.cc:2396] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:02.685140 2490 consensus_queue.cc:1046] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:02.686936 2430 raft_consensus.cc:3032] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:02.690840 2430 raft_consensus.cc:729] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:02.692082 2430 consensus_queue.cc:260] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.664831 2444 consensus_queue.cc:1035] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260619 21:16:02.715775 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 50723.2us
I20260619 21:16:02.746376 2491 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:02.759006 2491 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.793278 2334 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:02.797683 2491 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.797382 2334 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.800521 2534 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.803719 2534 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.811947 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.802601 2334 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.899128 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:02.908244 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:02.909518 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 497740us
I20260619 21:16:02.908159 1859 tablet_bootstrap.cc:654] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:02.914921 2334 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:02.910444 1816 raft_consensus.cc:3037] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:02.929186 2534 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:02.930104 2534 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.916185 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.915663 1812 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:02.938830 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:02.939970 1662 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:02.920619 2334 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.946440 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:02.946709 1635 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:02.950253 2412 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:02.950402 2430 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:02.950939 2412 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:02.951524 2412 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:02.955022 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:02.956120 1812 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:02.965708 1858 tablet_bootstrap.cc:654] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:02.944612 2334 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:02.974983 2261 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:02.981748 1635 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:16:02.974983 1710 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.544s user 0.023s sys 0.072s
W20260619 21:16:02.988397 1710 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.559s user 0.024s sys 0.073s
W20260619 21:16:02.988793 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 559421us
W20260619 21:16:03.091485 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 109797us
I20260619 21:16:03.092558 2538 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:03.093169 2538 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:03.093652 2538 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:02.990670 2261 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:03.106065 2261 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:03.106554 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:03.094568 2406 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:03.118743 2406 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.983585 2452 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.123208 2452 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.095263 2491 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:03.126181 2491 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:02.990361 2534 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.278228 2436 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.319257 2415 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:03.255754 2421 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.316082 2436 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.320161 2415 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.378973 1860 tablet_bootstrap.cc:492] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:03.379855 1860 ts_tablet_manager.cc:1397] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.431s user 0.023s sys 0.037s
I20260619 21:16:03.277143 2419 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:16:03.429486 1708 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.512s user 0.028s sys 0.031s
W20260619 21:16:03.429947 1708 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.513s user 0.028s sys 0.031s
W20260619 21:16:03.430265 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 516914us
I20260619 21:16:03.369758 2491 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:03.386387 2407 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s user 0.128s sys 0.160s
W20260619 21:16:03.426499 1744 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.667s user 0.015s sys 0.076s
W20260619 21:16:03.443591 1744 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.684s user 0.016s sys 0.076s
I20260619 21:16:03.459014 2491 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.463661 2491 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.466737 2491 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:03.447539 2407 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s user 0.128s sys 0.160s
W20260619 21:16:03.476133 2550 log.cc:924] Time spent T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.110s user 0.001s sys 0.000s
I20260619 21:16:03.476838 2404 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.436839 2419 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.367996 2421 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.440994 2406 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.488922 2461 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:03.562462 2412 consensus_meta.cc:287] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.610s user 0.010s sys 0.005s
I20260619 21:16:03.533270 2452 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.626055 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 383) took 1232 ms. Trace:
I20260619 21:16:03.626642 1817 rpcz_store.cc:276] 0619 21:16:02.393128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.395110 (+ 1982us) service_pool.cc:227] Handling call
0619 21:16:02.395429 (+ 319us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:02.395773 (+ 344us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:02.395801 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:02.396225 (+ 424us) log.cc:842] Serialized 26 byte log entry
0619 21:16:02.617936 (+221711us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:02.617996 (+ 60us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:02.618106 (+ 110us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:03.625705 (+1007599us) raft_consensus.cc:1671] finished
0619 21:16:03.625737 (+ 32us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:03.625960 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9329,"thread_start_us":194578,"threads_started":1,"wal-append.queue_time_us":582569}
I20260619 21:16:03.563108 2412 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.670460 2412 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:03.672458 2261 consensus_meta.cc:287] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.566s user 0.004s sys 0.008s
I20260619 21:16:03.673069 2261 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.675525 2421 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.676081 2261 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.682952 2412 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:03.683807 2412 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.685254 1665 raft_consensus.cc:1226] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260619 21:16:03.687000 2412 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:03.691939 2419 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.699630 2415 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.701123 2436 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:03.529268 2404 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.566301 2461 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:03.716451 2430 consensus_meta.cc:287] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.765s user 0.015s sys 0.000s
I20260619 21:16:03.704545 2404 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:03.724258 2538 consensus_meta.cc:287] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.630s user 0.007s sys 0.005s
I20260619 21:16:03.725697 1860 raft_consensus.cc:348] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.762348 1860 raft_consensus.cc:374] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:03.712766 2461 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:03.728031 1816 consensus_meta.cc:287] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.805s user 0.000s sys 0.014s
I20260619 21:16:03.763985 1860 raft_consensus.cc:729] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:03.659302 2546 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:03.768926 1860 consensus_queue.cc:260] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.552055 1660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 331) took 1858 ms. Trace:
I20260619 21:16:03.554824 2425 consensus_queue.cc:1035] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.220s
I20260619 21:16:03.761873 2553 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:03.766147 2412 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.767226 2341 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:03.773029 1634 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.509s user 0.021s sys 0.086s
W20260619 21:16:03.785053 1633 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.509s user 0.041s sys 0.070s
I20260619 21:16:03.788143 2412 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:03.789939 1634 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.526s user 0.022s sys 0.086s
I20260619 21:16:03.789788 2341 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.791954 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 363) took 1316 ms. Trace:
I20260619 21:16:03.772682 1816 raft_consensus.cc:2454] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:03.792363 2412 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:03.773016 2538 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.801733 1860 ts_tablet_manager.cc:1428] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.421s user 0.009s sys 0.000s
W20260619 21:16:03.825577 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 389985us
I20260619 21:16:03.795475 2261 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.801692 2452 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.834403 2452 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.835575 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:03.836477 1810 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:03.841113 2404 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.841922 2404 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.834282 2261 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.796855 2538 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.852299 2422 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:03.792542 1742 rpcz_store.cc:276] 0619 21:16:02.475878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.477287 (+ 1409us) service_pool.cc:227] Handling call
0619 21:16:03.790645 (+1313358us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:03.791022 (+ 377us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:03.791326 (+ 304us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:03.791394 (+ 68us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:03.791470 (+ 76us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:03.791538 (+ 68us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:03.791636 (+ 98us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:03.791861 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3812224}
I20260619 21:16:03.854553 2422 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.852319 2412 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.779814 1660 rpcz_store.cc:276] 0619 21:16:01.691824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.745475 (+ 53651us) service_pool.cc:227] Handling call
0619 21:16:01.746022 (+ 547us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:01.868242 (+122220us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.868298 (+ 56us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:01.869071 (+ 773us) log.cc:842] Serialized 26 byte log entry
0619 21:16:02.045230 (+176159us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:02.045286 (+ 56us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:03.442673 (+1397387us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003b3654. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:03.452262 (+ 9589us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:03.457346 (+ 5084us) raft_consensus.cc:1671] finished
0619 21:16:03.457371 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:03.550569 (+ 93198us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13649,"raft.queue_time_us":12146,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":26491,"spinlock_wait_cycles":279772416,"thread_start_us":155891,"threads_started":1,"wal-append.queue_time_us":228580,"wal-append.run_cpu_time_us":2990,"wal-append.run_wall_time_us":1314470}
I20260619 21:16:03.792876 2341 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:03.857770 2412 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.793805 2495 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:03.859552 2404 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:03.803129 1634 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.508s (SENT) [suppressed 2 similar messages]
I20260619 21:16:03.858345 2452 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.797035 2553 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.863508 2412 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:03.865422 2261 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.865007 2422 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.866142 2464 consensus_queue.cc:1046] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260619 21:16:03.868232 1635 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.605s user 0.033s sys 0.081s
W20260619 21:16:03.868583 1635 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.605s user 0.033s sys 0.081s
I20260619 21:16:03.876434 1668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 391) took 1480 ms. Trace:
I20260619 21:16:03.876955 1668 rpcz_store.cc:276] 0619 21:16:02.395709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.395859 (+ 150us) service_pool.cc:227] Handling call
0619 21:16:02.952799 (+556940us) spinlock_profiling.cc:243] Waited 974 us on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:02.963582 (+ 10783us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:02.974899 (+ 11317us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:02.974945 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:02.980719 (+ 5774us) log.cc:842] Serialized 26 byte log entry
0619 21:16:03.808674 (+827955us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:16:03.808734 (+ 60us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:03.808842 (+ 108us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:03.875985 (+ 67143us) raft_consensus.cc:1671] finished
0619 21:16:03.876015 (+ 30us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:03.876342 (+ 327us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":124505,"raft.queue_time_us":13356,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":2007,"spinlock_wait_cycles":2239232,"thread_start_us":465441,"threads_started":1,"wal-append.queue_time_us":775954}
I20260619 21:16:03.844303 2436 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:03.853031 1812 raft_consensus.cc:1261] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260619 21:16:03.920317 1635 leader_election.cc:302] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:03.781873 2546 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.937414 1859 tablet_bootstrap.cc:492] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:03.938117 1859 ts_tablet_manager.cc:1397] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 2.029s user 0.041s sys 0.065s
I20260619 21:16:03.943588 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:03.944782 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:03.972149 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:03.973443 1815 raft_consensus.cc:2379] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:03.960026 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:03.942513 2419 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:03.953310 2461 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:03.985224 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 477886us
I20260619 21:16:03.953445 2538 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:03.955618 2553 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.989641 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:03.965211 2452 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.999747 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:04.000563 1815 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:03.933802 2436 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:03.997465 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 194349us
I20260619 21:16:03.935563 2549 consensus_queue.cc:1035] T e3b2adcb38244b60bdd7ae5d5642f641 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260619 21:16:03.966898 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 372) took 1231 ms. Trace:
I20260619 21:16:03.965869 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:03.994935 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:03.995738 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:03.967329 2453 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:03.975960 2470 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.013811 2436 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:04.014010 2470 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.014549 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:04.014986 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:04.015583 1664 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:04.016026 2470 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.001585 2538 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:03.971175 2425 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.016364 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:04.016897 2425 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.018204 1817 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:03.989823 2419 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.018903 2425 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:03.993319 2267 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:04.019510 2267 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.020026 2419 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.020844 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:04.021528 2267 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:04.021914 2422 raft_consensus.cc:3032] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:04.022157 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:04.022325 2422 raft_consensus.cc:729] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:04.022375 2453 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.023275 2422 consensus_queue.cc:260] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.024471 2453 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.024534 2422 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:04.011691 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:04.012041 2452 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.013067 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:04.011659 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:03.982540 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.006888 2326 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.038278 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:04.026932 2553 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.051288 1817 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:04.029129 1818 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:04.035671 1740 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:04.003926 2559 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:04.060964 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:03.935812 2556 raft_consensus.cc:2781] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:04.018239 2538 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.029709 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:03.994637 2461 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.009261 2565 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:04.086509 1817 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:04.087157 1815 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:04.095075 1858 tablet_bootstrap.cc:492] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:04.095727 1858 ts_tablet_manager.cc:1397] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 1.619s user 0.018s sys 0.033s
I20260619 21:16:04.038545 1812 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:04.104519 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:04.063441 1738 raft_consensus.cc:1261] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:04.107028 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.115598 1814 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:04.105415 1817 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:04.088513 2527 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.092808 1742 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:04.063658 2453 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:04.124212 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.144408 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.145550 1669 raft_consensus.cc:2379] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:04.146471 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:04.147151 1669 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:04.124925 2577 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:04.149039 1818 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:04.126634 2546 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.129585 2527 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.062973 2326 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.131350 1665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 392) took 1735 ms. Trace:
I20260619 21:16:04.090847 2565 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.085951 2555 consensus_queue.cc:1035] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260619 21:16:04.139956 2452 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:04.165642 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 203431us
W20260619 21:16:04.156014 1818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 369) took 2099 ms (client timeout 1669 ms). Trace:
I20260619 21:16:04.063395 2553 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.105827 2556 raft_consensus.cc:686] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:04.160336 2461 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:04.161602 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:04.155625 2407 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:04.184988 1818 rpcz_store.cc:269] 0619 21:16:02.051064 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.054076 (+ 3012us) service_pool.cc:227] Handling call
0619 21:16:03.944291 (+1890215us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:04.150211 (+205920us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":23209856}
I20260619 21:16:04.163524 1859 raft_consensus.cc:348] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.164669 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.239070 1859 raft_consensus.cc:374] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:04.239437 1859 raft_consensus.cc:729] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:04.098941 2559 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:04.189034 2515 log.cc:924] Time spent T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.195s user 0.000s sys 0.001s
I20260619 21:16:04.188782 2261 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.237003 1743 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:04.240346 1859 consensus_queue.cc:260] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.246714 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 329) took 2555 ms. Trace:
I20260619 21:16:04.233446 2528 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.252995 2528 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.255343 2528 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:04.256179 2556 consensus_queue.cc:237] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.257913 2565 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.005358 1744 rpcz_store.cc:276] 0619 21:16:02.732613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.746744 (+ 14131us) service_pool.cc:227] Handling call
0619 21:16:02.758583 (+ 11839us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:03.444167 (+685584us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:03.444224 (+ 57us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:03.445208 (+ 984us) log.cc:842] Serialized 26 byte log entry
0619 21:16:03.894958 (+449750us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:03.934892 (+ 39934us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:03.949302 (+ 14410us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:03.960193 (+ 10891us) raft_consensus.cc:1671] finished
0619 21:16:03.963963 (+ 3770us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:03.964190 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":165445,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":4677,"thread_start_us":1123266,"threads_started":2,"wal-append.queue_time_us":424175}
I20260619 21:16:04.261024 2326 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.156798 2577 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.262730 2553 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.263535 2577 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.263370 2261 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.265343 2261 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.267916 1810 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
W20260619 21:16:04.268504 1810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 371) took 2216 ms (client timeout 1508 ms). Trace:
I20260619 21:16:04.270478 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:04.247925 1858 raft_consensus.cc:348] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.272425 1668 raft_consensus.cc:1226] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:04.287264 1858 raft_consensus.cc:374] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:04.287689 1858 raft_consensus.cc:729] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:16:04.157979 2453 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.291283 1860 tablet_bootstrap.cc:492] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:04.292339 1859 ts_tablet_manager.cc:1428] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.354s user 0.004s sys 0.004s
W20260619 21:16:04.271944 2570 log.cc:924] Time spent T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.246s user 0.000s sys 0.001s
I20260619 21:16:04.236649 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.242769 2407 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.556210 2553 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.573457 2553 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.575341 2407 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.576071 2553 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.576174 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:04.576894 2230 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:04.577613 2230 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.578375 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:04.579654 1634 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.011s sys 0.008s
W20260619 21:16:04.580009 1634 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.514s user 0.011s sys 0.008s
W20260619 21:16:04.580292 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 530949us
I20260619 21:16:04.418118 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:04.559912 2422 consensus_meta.cc:287] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.528s user 0.018s sys 0.016s
I20260619 21:16:04.562410 1669 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:04.247146 1666 rpcz_store.cc:276] 0619 21:16:01.691253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:01.743794 (+ 52541us) service_pool.cc:227] Handling call
0619 21:16:01.744482 (+ 688us) raft_consensus.cc:1457] Updating replica for 2 ops
0619 21:16:01.776328 (+ 31846us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:01.776395 (+ 67us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0619 21:16:01.793729 (+ 17334us) log.cc:842] Serialized 52 byte log entry
0619 21:16:02.045909 (+252180us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:02.099290 (+ 53381us) log.cc:842] Serialized 14 byte log entry
0619 21:16:02.166909 (+ 67619us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:02.170748 (+ 3839us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:02.704422 (+533674us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b1800211900. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb96764 00007f719bb99b11 00007f719bb99ac9 00007f719bb99a59 00007f719bb99a21 00007f719bb98d1d 00007f71a4dfa204 00007f71a4dfa039 00007f719bb78dc6
0619 21:16:03.942216 (+1237794us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b1800211900. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb96638 00007f719bb99b11 00007f719bb99ac9 00007f719bb99a59 00007f719bb99a21 00007f719bb98d1d 00007f71a4dfa204 00007f71a4dfa039 00007f719bb78dc6
0619 21:16:04.246431 (+304215us) raft_consensus.cc:1671] finished
0619 21:16:04.246460 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:04.246626 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35462,"prepare.queue_time_us":33917,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":3037,"raft.queue_time_us":70510,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":9180,"spinlock_wait_cycles":40985088,"thread_start_us":213924,"threads_started":1,"wal-append.queue_time_us":347707}
I20260619 21:16:04.290220 1858 consensus_queue.cc:260] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:04.158186 1665 rpcz_store.cc:276] 0619 21:16:02.396013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.398891 (+ 2878us) service_pool.cc:227] Handling call
0619 21:16:03.120097 (+721206us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:03.120417 (+ 320us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:03.685901 (+565484us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:04.130955 (+445054us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54001dbad4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:04.131258 (+ 303us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1312467584}
I20260619 21:16:04.565452 2406 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:04.268836 1810 rpcz_store.cc:269] 0619 21:16:02.051801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.054115 (+ 2314us) service_pool.cc:227] Handling call
0619 21:16:03.835345 (+1781230us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:04.268420 (+433075us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":22920448}
I20260619 21:16:04.591774 1742 raft_consensus.cc:2396] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:04.571038 2527 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.242511 1811 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:04.583688 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:04.586448 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 473549us
I20260619 21:16:04.572649 1660 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:04.679225 2555 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:04.602759 2406 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.051213 2555 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.052529 2406 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:04.786473 1814 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.662s user 0.018s sys 0.038s
W20260619 21:16:05.053231 1814 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.929s user 0.034s sys 0.045s
W20260619 21:16:04.604048 1707 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.501s user 0.015s sys 0.049s
W20260619 21:16:04.854902 1783 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.862s user 0.014s sys 0.022s
W20260619 21:16:05.054278 1783 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.061s user 0.022s sys 0.030s
W20260619 21:16:04.602739 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 510223us
W20260619 21:16:04.604198 2415 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s user 0.024s sys 0.040s
W20260619 21:16:05.055660 2544 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s user 0.017s sys 0.009s
W20260619 21:16:05.056056 2544 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s user 0.017s sys 0.009s
I20260619 21:16:05.056571 2544 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:04.845162 1567 rpcz_store.cc:275] Call kudu.master.MasterService.IsCreateTableDone from 127.0.0.1:37744 (request call id 41) took 1466 ms. Trace:
I20260619 21:16:05.057273 2544 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.057050 2553 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:04.780969 1564 catalog_manager.cc:5495] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe reported cstate change: term changed from 0 to 1, leader changed from <none> to d53146739e3240239c0a3d5ea90e07fe (127.0.99.2). New cstate: current_term: 1 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260619 21:16:04.895181 2417 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.868s user 0.002s sys 0.011s
W20260619 21:16:04.778148 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:04.619834 2424 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:04.575309 2453 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:04.745808 2341 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.681s user 0.013s sys 0.014s
I20260619 21:16:04.684044 1663 raft_consensus.cc:1261] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:04.688635 2349 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:04.689292 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:04.895574 2464 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.777s user 0.000s sys 0.014s
W20260619 21:16:04.686987 1664 consensus_meta.cc:287] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.671s user 0.011s sys 0.001s
W20260619 21:16:05.061029 2421 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s user 0.009s sys 0.026s
I20260619 21:16:04.776832 1565 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37710 (request call id 82) took 6663 ms. Trace:
W20260619 21:16:04.759339 2458 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s user 0.012s sys 0.018s
I20260619 21:16:05.044907 2581 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:16:04.893123 2556 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s user 0.010s sys 0.002s
I20260619 21:16:04.603411 2425 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.047638 2261 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:04.686519 1859 tablet_bootstrap.cc:492] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:04.616971 1858 ts_tablet_manager.cc:1428] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.521s user 0.020s sys 0.016s
W20260619 21:16:04.729067 2495 consensus_meta.cc:287] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.870s user 0.015s sys 0.000s
I20260619 21:16:04.960067 2559 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:04.600620 2422 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.067885 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 351) took 1101 ms. Trace:
I20260619 21:16:05.107208 1565 rpcz_store.cc:276] 0619 21:15:58.127403 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:58.139947 (+ 12544us) service_pool.cc:227] Handling call
0619 21:15:58.494153 (+354206us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b5400370ad4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43af0 00007f719eac6f2a 00007f71a4a18d6c 00007f71a4b12c3e 00007f71a2e24fb4 00007f71a2e24f42 00007f71a2e24e71 00007f71a2e24dec 00007f71a2e240a2 00007f719cee894c 00007f719cee7d86
0619 21:16:04.776730 (+6282577us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:16:01.757325 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:16:01.758115 (+ 790us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:16:01.758163 (+ 48us) write_op.cc:435] Acquired schema lock
0619 21:16:01.758175 (+ 12us) tablet.cc:660] Decoding operations
0619 21:16:01.758657 (+ 482us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:16:01.758752 (+ 95us) write_op.cc:641] Partition lock acquired for write op
0619 21:16:01.758791 (+ 39us) tablet.cc:683] Acquiring locks for 1 operations
0619 21:16:01.759057 (+ 266us) tablet.cc:699] Row locks acquired
0619 21:16:01.759069 (+ 12us) write_op.cc:260] PREPARE: finished
0619 21:16:01.759271 (+ 202us) write_op.cc:270] Start()
0619 21:16:01.759428 (+ 157us) write_op.cc:276] Timestamp: P: 1781903761759220 usec, L: 0
0619 21:16:01.759444 (+ 16us) op_driver.cc:331] REPLICATION: starting
0619 21:16:01.794548 (+ 35104us) log.cc:842] Serialized 531 byte log entry
0619 21:16:02.065247 (+270699us) op_driver.cc:444] REPLICATION: finished
0619 21:16:02.246190 (+180943us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719bb785fe 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f71a24ce410
0619 21:16:02.397791 (+151601us) write_op.cc:301] APPLY: starting
0619 21:16:02.397901 (+ 110us) tablet.cc:1365] starting BulkCheckPresence
0619 21:16:02.401458 (+ 3557us) tablet.cc:1368] finished BulkCheckPresence
0619 21:16:02.403413 (+ 1955us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:16:02.403661 (+ 248us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:16:02.403827 (+ 166us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0619 21:16:02.403870 (+ 43us) write_op.cc:312] APPLY: finished
0619 21:16:02.404068 (+ 198us) log.cc:842] Serialized 22 byte log entry
0619 21:16:02.404233 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:02.404317 (+ 84us) write_op.cc:454] Released schema lock
0619 21:16:02.404353 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0619 21:16:02.404570 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:02.404602 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10450,"reported_tablets":8,"spinlock_wait_cycles":194485120,"child_traces":[["op",{"apply.queue_time_us":327953,"apply.run_cpu_time_us":1720,"apply.run_wall_time_us":11363,"num_ops":1,"prepare.queue_time_us":212822,"prepare.run_cpu_time_us":3736,"prepare.run_wall_time_us":48328,"replication_time_us":305680,"spinlock_wait_cycles":13950848,"thread_start_us":397293,"threads_started":2}]]}
W20260619 21:16:05.109902 2556 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s user 0.010s sys 0.006s
I20260619 21:16:04.746325 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:05.066980 1567 rpcz_store.cc:276] 0619 21:16:03.311976 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:03.312513 (+ 537us) service_pool.cc:227] Handling call
0619 21:16:03.321558 (+ 9045us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0619 21:16:04.264264 (+942706us) catalog_manager.cc:2245] Verify if the table creation is in progress for test-workload [id=1995c89d987e45a6af7b17d850db3dbb]
0619 21:16:04.778425 (+514161us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1455488}
W20260619 21:16:05.075933 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 494912us
I20260619 21:16:05.127707 1634 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:05.090349 1783 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:04.785331 2463 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s user 0.009s sys 0.018s
I20260619 21:16:04.626255 1662 raft_consensus.cc:2379] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:05.100582 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:04.596487 2452 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.049692 2230 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.097729 1667 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:05.136092 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:05.137295 1667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 355) took 1169 ms (client timeout 1502 ms). Trace:
I20260619 21:16:05.136325 2474 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260619 21:16:05.137719 1667 rpcz_store.cc:269] 0619 21:16:03.967788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:03.980750 (+ 12962us) service_pool.cc:227] Handling call
0619 21:16:04.689105 (+708355us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.137234 (+448129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":21190016}
I20260619 21:16:05.110705 2555 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.120451 2551 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.091956 2553 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:05.092881 2417 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.086s user 0.007s sys 0.011s
I20260619 21:16:05.073567 2574 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.091465 2444 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:16:05.098620 2341 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.037s user 0.013s sys 0.024s
W20260619 21:16:05.114359 1858 ts_tablet_manager.cc:1460] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: Tablet startup took 1018ms
W20260619 21:16:05.100878 2464 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.996s user 0.000s sys 0.018s
I20260619 21:16:05.102285 2586 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:05.109016 2421 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s user 0.009s sys 0.026s
W20260619 21:16:05.108073 2458 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s user 0.016s sys 0.024s
I20260619 21:16:05.093708 1660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 358) took 1102 ms. Trace:
I20260619 21:16:05.122588 1669 rpcz_store.cc:276] 0619 21:16:03.965982 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.066951 (+1100969us) service_pool.cc:227] Handling call
0619 21:16:05.067315 (+ 364us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:05.067492 (+ 177us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:05.067515 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:05.067562 (+ 47us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:05.067594 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:05.067672 (+ 78us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:05.067810 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:05.153409 1783 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:05.068790 1782 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.820s user 0.002s sys 0.007s
W20260619 21:16:05.164929 1782 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.916s user 0.003s sys 0.007s
I20260619 21:16:05.125674 2422 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.134209 1668 raft_consensus.cc:2379] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:05.166622 1668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 361) took 1158 ms. Trace:
I20260619 21:16:05.166692 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.167083 1668 rpcz_store.cc:276] 0619 21:16:04.008033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:04.585105 (+577072us) service_pool.cc:227] Handling call
0619 21:16:05.166538 (+581433us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:05.134588 1634 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:05.107936 1664 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:05.090826 2527 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.137624 1742 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:05.138543 2559 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:05.111407 2425 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.170990 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.093993 1811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 408) took 1140 ms. Trace:
I20260619 21:16:05.113108 1663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 339) took 2700 ms. Trace:
W20260619 21:16:05.134985 2463 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.994s user 0.015s sys 0.029s
I20260619 21:16:05.098484 2424 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.193143 2424 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.193761 2424 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:05.194136 2424 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.194516 2424 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:05.195880 1860 tablet_bootstrap.cc:654] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:05.196261 2422 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.112819 2453 consensus_queue.cc:1035] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260619 21:16:05.162374 2586 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.138326 2474 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:05.197731 2474 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:16:05.165361 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 994590us
W20260619 21:16:05.189865 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 55370.2us
W20260619 21:16:05.140141 1858 ts_tablet_manager.cc:1462] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe: Trace:
0619 21:15:42.938628 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:42.938895 (+ 267us) service_pool.cc:227] Handling call
0619 21:15:42.940692 (+ 1797us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:15:42.940800 (+ 108us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:15:43.077985 (+137185us) tablet_metadata.cc:641] Metadata flushed
0619 21:15:43.171157 (+ 93172us) tablet_replica.cc:205] Creating consensus instance
0619 21:15:43.186700 (+ 15543us) inbound_call.cc:173] Queueing success response
0619 21:16:02.471603 (+19284903us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:16:04.096064 (+1624461us) ts_tablet_manager.cc:1429] Starting tablet replica
0619 21:16:04.246063 (+149999us) tablet_replica.cc:259] Starting instrumentation
0619 21:16:04.246587 (+ 524us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":13547,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":46102,"tablet-open.queue_time_us":19287406}
I20260619 21:16:05.121760 2261 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.114498 2581 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.190531 1661 raft_consensus.cc:1226] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:05.172115 1811 rpcz_store.cc:276] 0619 21:16:03.944640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:03.953069 (+ 8429us) service_pool.cc:227] Handling call
0619 21:16:04.161368 (+208299us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.084549 (+923181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":25006336}
I20260619 21:16:05.190909 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.190104 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.149642 1662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 360) took 1137 ms. Trace:
I20260619 21:16:05.218863 1742 raft_consensus.cc:2379] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:05.219674 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:05.221329 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 129872us
I20260619 21:16:05.223171 1635 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.220203 1742 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:05.227499 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:05.138650 2452 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.230545 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.231442 1742 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:05.142704 2551 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.164055 1660 rpcz_store.cc:276] 0619 21:16:03.982127 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:04.123519 (+141392us) service_pool.cc:227] Handling call
0619 21:16:05.084623 (+961104us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:05.198884 2586 leader_election.cc:288] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.234037 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.210397 2598 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:05.174958 1663 rpcz_store.cc:276] 0619 21:16:02.412522 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.487887 (+ 75365us) service_pool.cc:227] Handling call
0619 21:16:04.683171 (+2195284us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:04.683438 (+ 267us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:05.098355 (+414917us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:05.113039 (+ 14684us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":34789632}
I20260619 21:16:05.149677 2444 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.238893 1783 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:05.152874 2574 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.247651 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.248296 1741 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:05.175506 2527 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.240262 1635 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.243008 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:05.217520 1664 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 335) took 3115 ms (client timeout 2055 ms). Trace:
I20260619 21:16:05.211553 1662 rpcz_store.cc:276] 0619 21:16:04.004746 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:04.577917 (+573171us) service_pool.cc:227] Handling call
0619 21:16:05.142230 (+564313us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:05.256465 1635 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.252271 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.253175 2444 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:05.254041 2601 log.cc:924] Time spent T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260619 21:16:05.254458 2544 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.239714 2453 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.243119 2600 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:05.262241 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.262668 1635 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.251351 1740 raft_consensus.cc:2396] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:05.244428 2602 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.261749 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:05.274155 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:05.267886 1815 raft_consensus.cc:1226] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260619 21:16:05.268355 1814 raft_consensus.cc:1226] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:05.269142 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.273002 1635 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.278618 1635 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.279814 1635 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.282878 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.286284 1742 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:05.256167 2598 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:05.271116 2555 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:05.291129 1814 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:05.260449 2406 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.310986 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.271739 2600 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.243655 2592 consensus_queue.cc:1035] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260619 21:16:05.296352 2261 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.295264 2607 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.295384 2230 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:05.289732 2602 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.320225 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.313632 2604 raft_consensus.cc:2781] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:05.289608 2556 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:05.312424 2544 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.297890 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 394) took 2894 ms. Trace:
I20260619 21:16:05.288668 2599 raft_consensus.cc:2781] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:05.299124 2598 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.302995 2555 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.281909 2453 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.473788 1859 tablet_bootstrap.cc:700] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Time spent opening tablet: real 0.187s user 0.002s sys 0.000s
I20260619 21:16:05.473026 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:05.487258 1742 raft_consensus.cc:2362] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1. Re-sending same reply.
I20260619 21:16:05.486822 1859 tablet_bootstrap.cc:654] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
W20260619 21:16:05.401235 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 162388us
W20260619 21:16:05.251257 1664 rpcz_store.cc:269] 0619 21:16:02.059778 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.062209 (+ 2431us) service_pool.cc:227] Handling call
0619 21:16:02.501576 (+439367us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:04.014847 (+1513271us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.171624 (+1156777us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":62030592}
I20260619 21:16:05.490952 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.316670 2406 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.536180 1783 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:05.341893 2594 log.cc:924] Time spent T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.138s user 0.000s sys 0.000s
I20260619 21:16:05.322201 2600 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:05.534358 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 254602us
I20260619 21:16:05.466672 2456 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.427685 2581 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.336383 2230 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:05.416481 2574 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.378575 2586 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.384094 2422 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.347697 2261 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:05.466878 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.540172 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.436357 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 359) took 1442 ms. Trace:
I20260619 21:16:05.341888 2452 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.496362 2555 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:05.545135 1738 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:05.579873 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.514066 2556 raft_consensus.cc:686] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:05.512451 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.587033 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.529723 2599 raft_consensus.cc:686] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [term 5 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:05.488682 2604 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.490846 2598 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:05.541785 1635 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:05.336369 2553 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.565759 1666 rpcz_store.cc:276] 0619 21:16:03.993573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.435090 (+1441517us) service_pool.cc:227] Handling call
0619 21:16:05.435614 (+ 524us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:05.435854 (+ 240us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:05.435903 (+ 49us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:05.435961 (+ 58us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:05.436014 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:05.436103 (+ 89us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:05.436270 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:05.584154 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.585275 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.586185 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:05.586535 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.586364 2424 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.420751 2474 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.498754 2544 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.555743 2230 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:05.345180 2607 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.547008 2444 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.546799 2527 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.560561 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.546595 2456 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.561915 2586 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.560774 2422 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.573609 1816 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:05.560035 2261 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.593005 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.738659 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:05.737931 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 196233us
I20260619 21:16:05.741742 1741 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:05.742235 1814 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:05.743669 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:05.744259 1814 raft_consensus.cc:2362] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4. Re-sending same reply.
I20260619 21:16:05.745052 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.746310 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:05.747246 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.747946 1815 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:05.751279 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:05.747925 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 402) took 1126 ms. Trace:
I20260619 21:16:05.752738 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.754622 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.756264 1860 tablet_bootstrap.cc:492] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:05.570099 2610 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:05.747044 1813 raft_consensus.cc:2362] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3. Re-sending same reply.
I20260619 21:16:05.753278 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.757673 2490 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:05.747102 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.761090 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.753898 1743 rpcz_store.cc:276] 0619 21:16:04.622058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:04.676875 (+ 54817us) service_pool.cc:227] Handling call
0619 21:16:05.437386 (+760511us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.586012 (+148626us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.747854 (+161842us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":938266752}
I20260619 21:16:05.767031 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.748273 2261 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:05.590682 2460 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.762244 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.780622 1817 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:05.573609 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.769958 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.660025 2421 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:05.652871 2606 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.714689 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.774432 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.792416 1815 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:05.757980 2527 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.690331 2602 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.756871 1860 ts_tablet_manager.cc:1397] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.474s user 0.030s sys 0.047s
I20260619 21:16:05.714182 2406 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.773620 2490 raft_consensus.cc:686] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:05.808003 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.653517 2612 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.602574 2604 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:05.679937 2349 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.689410 2453 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.714547 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.685947 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.741258 2592 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.782015 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.787490 1818 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:05.752676 2574 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.770179 2610 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:05.760597 2474 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.755828 1740 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:05.769731 2452 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.760818 2559 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.763927 1739 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:05.909864 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.488425 1661 rpcz_store.cc:276] 0619 21:16:02.407416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.409656 (+ 2240us) service_pool.cc:227] Handling call
0619 21:16:04.050293 (+1640637us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:05.172148 (+1121855us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:05.231100 (+ 58952us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:05.297672 (+ 66572us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540036db50. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabda09 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:05.297817 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2503906816}
I20260619 21:16:05.912797 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.913846 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:05.915092 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 157395us
I20260619 21:16:05.916008 1662 raft_consensus.cc:2396] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:05.917939 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:05.919199 1663 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:05.920153 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:05.920838 1782 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:05.921145 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:05.827540 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:05.923069 1741 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:05.924567 1782 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:05.926110 1736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 394) took 1342 ms (client timeout 1534 ms). Trace:
W20260619 21:16:05.926704 1736 rpcz_store.cc:269] 0619 21:16:04.583941 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:04.585940 (+ 1999us) service_pool.cc:227] Handling call
0619 21:16:04.680800 (+ 94860us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.337707 (+656907us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.919745 (+582038us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.926006 (+ 6261us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1003567744}
I20260619 21:16:05.913975 1744 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
W20260619 21:16:05.805519 1783 leader_election.cc:341] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:16:05.929741 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 124099us
I20260619 21:16:05.809338 1735 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
W20260619 21:16:05.787216 1737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 396) took 1194 ms (client timeout 1532 ms). Trace:
W20260619 21:16:05.932633 1737 rpcz_store.cc:269] 0619 21:16:04.584725 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:04.586100 (+ 1375us) service_pool.cc:227] Handling call
0619 21:16:04.690544 (+104444us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.422903 (+732359us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.584015 (+161112us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:05.778881 (+194866us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":141073408}
I20260619 21:16:05.808853 1743 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:05.826637 1817 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:05.789336 1811 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:05.820210 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.938169 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.938629 1813 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:05.822391 2349 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.791755 2421 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.951452 2421 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:05.953258 2537 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:05.954142 2537 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.818499 2424 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.748055 2444 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.958276 1742 raft_consensus.cc:2454] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:05.832396 2452 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.959425 1812 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:05.956899 1858 tablet_bootstrap.cc:492] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:05.794483 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.961911 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 398) took 1361 ms. Trace:
I20260619 21:16:05.962309 2452 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.788736 2460 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.839937 2474 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:05.859112 2490 consensus_queue.cc:237] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.809151 2527 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.864614 2544 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.807718 2553 consensus_queue.cc:1046] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260619 21:16:05.828585 2555 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.687218 2556 consensus_queue.cc:237] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.832623 2559 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.809345 2564 consensus_queue.cc:1046] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260619 21:16:05.832648 2574 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.809777 2585 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:05.863446 2586 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.829445 2592 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.879436 2598 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.872733 2600 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.654908 2599 consensus_queue.cc:237] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.795168 2606 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.863873 2607 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.829655 2610 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:05.821935 2612 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.841068 2613 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:05.879268 2604 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:05.981979 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.950062 2456 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:05.956609 2422 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.963595 2349 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.960533 2444 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:05.970177 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 185909us
I20260619 21:16:05.974663 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.962491 1738 rpcz_store.cc:276] 0619 21:16:04.600506 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.921852 (+1321346us) service_pool.cc:227] Handling call
0619 21:16:05.960613 (+ 38761us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:05.961077 (+ 464us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:05.961125 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:05.961234 (+ 109us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:05.961292 (+ 58us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:05.961427 (+ 135us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:05.961633 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:05.980517 2623 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:05.968815 2460 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.949399 1783 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:06.035652 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 102814us
I20260619 21:16:06.037786 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.038935 1665 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:06.038800 2585 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.040365 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.009332 1660 raft_consensus.cc:1226] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:06.009171 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.010794 2564 raft_consensus.cc:3032] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:06.011622 2598 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.011751 2474 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.012295 2592 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.013093 2424 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:06.013945 2341 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:06.016651 2559 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.017287 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.017740 2555 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:05.970146 2621 consensus_queue.cc:1035] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260619 21:16:06.024245 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.026373 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.975639 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:05.969812 2620 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.073205 1669 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:06.044237 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.046952 2574 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:16:06.047703 2613 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.060546 1859 tablet_bootstrap.cc:492] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:06.014482 1860 raft_consensus.cc:348] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.077302 1860 raft_consensus.cc:374] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:06.078056 2623 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.078284 1662 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:06.078567 2623 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.079002 2623 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.079173 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.079421 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:06.080806 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 262541us
I20260619 21:16:06.064787 2564 raft_consensus.cc:729] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:06.077585 1860 raft_consensus.cc:729] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:06.081597 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.068066 2349 raft_consensus.cc:3032] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:06.113225 1667 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:06.110090 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.117966 2564 consensus_queue.cc:260] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.119486 2230 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.121248 1817 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:06.124064 2261 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.125065 2604 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.126157 2527 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.127316 2474 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.074290 1810 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:06.128532 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.108047 2349 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:06.135900 2612 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.136106 2349 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.137677 2600 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.137849 2349 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:06.141374 1741 raft_consensus.cc:2454] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:06.146301 1859 ts_tablet_manager.cc:1397] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 1.460s user 0.047s sys 0.031s
I20260619 21:16:06.147693 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:06.120436 2421 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.132959 1860 consensus_queue.cc:260] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.152477 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.133870 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.131760 2564 raft_consensus.cc:3037] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:06.130126 2585 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.160694 2425 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.074345 2341 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.130671 2613 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.164527 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.155431 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.142525 1665 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:06.162242 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.169636 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 399) took 1566 ms. Trace:
I20260619 21:16:06.170081 1736 rpcz_store.cc:276] 0619 21:16:04.600828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.928292 (+1327464us) service_pool.cc:227] Handling call
0619 21:16:06.032081 (+103789us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.129690 (+ 97609us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.130352 (+ 662us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.133929 (+ 3577us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:06.155658 (+ 21729us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.162099 (+ 6441us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.167260 (+ 5161us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:06.170814 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.171380 2551 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.177208 2425 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:06.129990 2424 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.183904 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.187408 1669 raft_consensus.cc:2379] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:06.188726 1741 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:06.190857 2606 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.192188 2551 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:06.183449 2425 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.192891 2551 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.194890 2425 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.195031 2551 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.183663 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.204427 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.206737 2490 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:06.209787 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.210134 2490 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.211750 2341 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.212452 2490 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.213336 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.214663 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.216064 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.217234 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.218704 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.219511 1663 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:06.208500 2527 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:06.202172 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.221103 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.221725 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.221922 1809 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:06.221683 2527 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.223414 1708 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.224186 2527 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.225570 1710 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.225818 2424 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.226464 2422 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:06.226640 1710 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:06.227731 2553 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.228193 2553 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.228534 2553 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.229271 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.220835 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.228677 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.375281 1860 ts_tablet_manager.cc:1428] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.572s user 0.015s sys 0.004s
I20260619 21:16:06.376686 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.235651 2230 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.351303 1634 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:06.384219 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 50886.5us
W20260619 21:16:06.380973 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 64605.4us
I20260619 21:16:06.379487 1669 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:06.223068 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:06.381429 1738 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:06.389196 1663 raft_consensus.cc:2379] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:06.389603 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.390188 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.390947 1663 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:06.391887 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 406) took 1174 ms. Trace:
I20260619 21:16:06.294414 2625 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.392307 1738 rpcz_store.cc:276] 0619 21:16:05.217781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.390698 (+1172917us) service_pool.cc:227] Handling call
0619 21:16:06.391140 (+ 442us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.391507 (+ 367us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.391540 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.391618 (+ 78us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:06.391652 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.391711 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.391829 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:06.393164 2625 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:06.393616 2625 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.394284 1738 raft_consensus.cc:1261] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:06.394963 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 407) took 1169 ms. Trace:
I20260619 21:16:06.395327 1738 rpcz_store.cc:276] 0619 21:16:05.225215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.393466 (+1168251us) service_pool.cc:227] Handling call
0619 21:16:06.393834 (+ 368us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.394666 (+ 832us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.394907 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:06.397466 1813 raft_consensus.cc:1226] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: [2.1-2.1]
I20260619 21:16:06.398947 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.399407 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.401347 1708 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.401351 2421 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.401924 2628 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:06.402745 1667 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:06.403029 2425 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:06.403308 1744 raft_consensus.cc:1226] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:06.391924 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.404039 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 403) took 1676 ms. Trace:
I20260619 21:16:06.405260 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:06.378995 1860 ts_tablet_manager.cc:1460] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Tablet startup took 575ms
I20260619 21:16:06.403787 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.404819 1708 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.324224 1710 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.358773 2261 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.401265 1782 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.233567 1809 raft_consensus.cc:3037] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:06.381523 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 402) took 1179 ms. Trace:
W20260619 21:16:06.418547 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 187598us
I20260619 21:16:06.420197 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.405911 2537 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.349184 2610 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.408234 1859 raft_consensus.cc:348] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.426147 1859 raft_consensus.cc:374] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:06.426501 1859 raft_consensus.cc:729] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
W20260619 21:16:06.405906 1739 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 380) took 4211 ms (client timeout 1669 ms). Trace:
I20260619 21:16:06.427286 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:06.415714 2628 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:06.431646 1667 raft_consensus.cc:2362] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4. Re-sending same reply.
I20260619 21:16:06.427434 1710 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.435959 2613 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:06.436664 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.437083 1818 raft_consensus.cc:2379] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:06.437541 1811 raft_consensus.cc:2354] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
W20260619 21:16:06.438261 2635 log.cc:924] Time spent T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.208s user 0.001s sys 0.000s
I20260619 21:16:06.438567 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:06.411263 1744 rpcz_store.cc:276] 0619 21:16:04.727896 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.928557 (+1200661us) service_pool.cc:227] Handling call
0619 21:16:06.402867 (+474310us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:06.403085 (+ 218us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.403758 (+ 673us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.403966 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":56781568}
W20260619 21:16:06.439517 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 88385.4us
I20260619 21:16:06.440429 2610 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.441045 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 425) took 1102 ms. Trace:
I20260619 21:16:06.441406 1744 rpcz_store.cc:276] 0619 21:16:05.338285 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.439945 (+1101660us) service_pool.cc:227] Handling call
0619 21:16:06.440362 (+ 417us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.440638 (+ 276us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.440666 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.440745 (+ 79us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:06.440777 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.440839 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.440987 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:06.430353 1740 raft_consensus.cc:2396] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:06.445031 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:06.447190 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.447280 1634 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.447992 1669 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:06.448423 2599 raft_consensus.cc:2781] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:06.448829 2599 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:06.449028 2551 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.449261 2599 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:06.449440 2551 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.449888 2551 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.450268 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 428) took 1111 ms. Trace:
I20260619 21:16:06.450675 1742 rpcz_store.cc:276] 0619 21:16:05.339199 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.441959 (+1102760us) service_pool.cc:227] Handling call
0619 21:16:06.442326 (+ 367us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.442562 (+ 236us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.442593 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.442659 (+ 66us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:06.442700 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.442767 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.450200 (+ 7433us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:06.427691 2421 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:06.445308 1708 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.445523 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.436571 2613 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:06.430270 1739 rpcz_store.cc:269] 0619 21:16:02.194836 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.197700 (+ 2864us) service_pool.cc:227] Handling call
0619 21:16:06.405504 (+4207804us) spinlock_profiling.cc:243] Waited 3.29 s on lock 0x7b540040e554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:06.405679 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7569338752}
I20260619 21:16:06.430476 1859 consensus_queue.cc:260] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.438477 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:06.458446 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.457504 1663 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
W20260619 21:16:06.462366 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 61206.5us
I20260619 21:16:06.462503 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.425539 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.465134 1665 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:06.464794 2623 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.465325 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.466027 1663 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:06.467128 2623 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.468390 2620 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:06.468478 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:06.420163 1736 rpcz_store.cc:276] 0619 21:16:05.202208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.179596 (+977388us) service_pool.cc:227] Handling call
0619 21:16:06.228245 (+ 48649us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.295325 (+ 67080us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.295383 (+ 58us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.295472 (+ 89us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:06.317531 (+ 22059us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.352773 (+ 35242us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.378936 (+ 26163us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:06.469503 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.464326 1668 raft_consensus.cc:2454] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:16:06.476827 2349 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:06.477129 1663 raft_consensus.cc:1261] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:06.477353 2349 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:06.477777 2349 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:06.478541 1663 raft_consensus.cc:1226] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
W20260619 21:16:06.484529 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 59778us
I20260619 21:16:06.487044 1710 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:06.406924 1860 ts_tablet_manager.cc:1462] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Trace:
0619 21:15:42.706136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:15:43.126873 (+420737us) service_pool.cc:227] Handling call
0619 21:15:43.128642 (+ 1769us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:15:43.128838 (+ 196us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:15:43.215521 (+ 86683us) tablet_metadata.cc:641] Metadata flushed
0619 21:15:43.317258 (+101737us) tablet_replica.cc:205] Creating consensus instance
0619 21:15:43.321925 (+ 4667us) inbound_call.cc:173] Queueing success response
0619 21:16:04.252899 (+20930974us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:16:05.803329 (+1550430us) ts_tablet_manager.cc:1429] Starting tablet replica
0619 21:16:05.954069 (+150740us) tablet_replica.cc:259] Starting instrumentation
0619 21:16:05.955340 (+ 1271us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":44908,"raft.queue_time_us":13539,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":11830,"tablet-open.queue_time_us":20932159}
I20260619 21:16:06.498379 1860 tablet_bootstrap.cc:492] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:06.498703 2613 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.500350 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.469871 1813 raft_consensus.cc:2354] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:06.480788 1663 raft_consensus.cc:1226] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:06.508481 1782 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:06.506866 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 61678.7us
I20260619 21:16:06.542219 2465 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.542774 2465 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.543141 2465 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.511221 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:06.547278 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.548062 1818 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:06.549124 1635 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.552440 1860 tablet_bootstrap.cc:654] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:06.553427 1634 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:06.554383 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 68066.5us
I20260619 21:16:06.555264 1710 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.556342 1710 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.555981 2599 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.564577 1859 ts_tablet_manager.cc:1428] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.418s user 0.014s sys 0.005s
I20260619 21:16:06.560309 2460 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:06.593717 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 424) took 1080 ms. Trace:
I20260619 21:16:06.595728 2628 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:06.598141 1859 tablet_bootstrap.cc:492] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:06.602478 2559 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
W20260619 21:16:06.602828 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 92658.9us
I20260619 21:16:06.603374 2581 raft_consensus.cc:2781] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
W20260619 21:16:06.608431 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 100007us
I20260619 21:16:06.608718 1708 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:06.611056 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 151347us
I20260619 21:16:06.602836 2599 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.609995 1858 tablet_bootstrap.cc:654] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:06.592458 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:06.573433 2424 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:06.579245 2537 raft_consensus.cc:2781] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:06.621701 2460 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:06.643335 2349 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.645097 2425 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260619 21:16:06.647009 1710 leader_election.cc:341] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:16:06.649228 1635 leader_election.cc:341] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:06.649559 2581 raft_consensus.cc:686] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:06.622763 1810 rpcz_store.cc:276] 0619 21:16:05.513092 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.523513 (+ 10421us) service_pool.cc:227] Handling call
0619 21:16:05.585130 (+ 61617us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:06.593410 (+1008280us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54001162d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:06.593610 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":718905600}
I20260619 21:16:06.650951 2460 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:06.636199 2537 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:16:06.640480 1782 leader_election.cc:341] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:06.659339 1782 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.638845 2559 raft_consensus.cc:686] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:06.676354 1635 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:06.657016 2425 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.699962 2625 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.661082 2537 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:06.671073 2553 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.702698 2425 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:06.725785 1634 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:06.730792 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 93106.5us
W20260619 21:16:06.733557 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 70811.2us
I20260619 21:16:06.735293 1782 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.719267 2606 raft_consensus.cc:2726] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:06.751224 2625 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:06.753033 1708 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:06.753154 2581 consensus_queue.cc:237] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.754491 1708 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.754732 2559 consensus_queue.cc:237] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.755414 1708 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:06.760210 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 48776us
I20260619 21:16:06.749248 2621 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260619 21:16:06.765581 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 100243us
I20260619 21:16:06.770385 2453 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:06.770924 2453 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:06.771369 2453 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:06.772003 1735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 397) took 1596 ms. Trace:
I20260619 21:16:06.790232 2406 raft_consensus.cc:2781] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:06.790793 2406 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:06.791232 2406 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:06.768361 2625 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:06.792784 2625 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.793231 2625 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:06.794615 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 411) took 1265 ms. Trace:
I20260619 21:16:06.794806 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:06.795603 1662 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:06.766139 2621 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:06.797251 2621 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:06.797358 2652 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.797940 2652 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.798365 2652 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.799053 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:06.768107 2551 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.802014 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 423) took 1304 ms. Trace:
I20260619 21:16:06.802488 1812 rpcz_store.cc:276] 0619 21:16:05.498000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.512446 (+ 14446us) service_pool.cc:227] Handling call
0619 21:16:06.801674 (+1289228us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54003430d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:06.801925 (+ 251us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":501820672}
I20260619 21:16:06.803890 2551 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.805040 2349 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.805132 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.795069 1744 rpcz_store.cc:276] 0619 21:16:05.529075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.442416 (+913341us) service_pool.cc:227] Handling call
0619 21:16:06.442911 (+ 495us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.443288 (+ 377us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:06.564289 (+121001us) log.cc:842] Serialized 14 byte log entry
0619 21:16:06.773981 (+209692us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.779176 (+ 5195us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:06.794300 (+ 15124us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.794389 (+ 89us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.794548 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":56813,"prepare.queue_time_us":198689,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":41466,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":185571,"threads_started":2}
I20260619 21:16:06.807119 1737 raft_consensus.cc:2354] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:06.807507 2621 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.809767 2553 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.810585 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 411) took 1638 ms. Trace:
I20260619 21:16:06.811000 1740 rpcz_store.cc:276] 0619 21:16:05.172634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.142253 (+969619us) service_pool.cc:227] Handling call
0619 21:16:06.810372 (+668119us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002b8254. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:06.810517 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":596626304}
I20260619 21:16:06.814646 1860 tablet_bootstrap.cc:492] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:06.815219 1860 ts_tablet_manager.cc:1397] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.317s user 0.020s sys 0.004s
I20260619 21:16:06.811360 2453 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.817689 1741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 403) took 1615 ms. Trace:
I20260619 21:16:06.783299 1782 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.818241 1741 rpcz_store.cc:276] 0619 21:16:05.202482 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.257972 (+1055490us) service_pool.cc:227] Handling call
0619 21:16:06.316890 (+ 58918us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.428709 (+111819us) raft_consensus.cc:1520] Early marking committed up to index 3
0619 21:16:06.624722 (+196013us) log.cc:842] Serialized 14 byte log entry
0619 21:16:06.817199 (+192477us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.817269 (+ 70us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:06.817324 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.817455 (+ 131us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.817614 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":149902,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"thread_start_us":244449,"threads_started":2}
I20260619 21:16:06.826491 2424 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:06.829412 2621 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.837913 1635 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.772408 1735 rpcz_store.cc:276] 0619 21:16:05.175406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:05.931877 (+756471us) service_pool.cc:227] Handling call
0619 21:16:06.770842 (+838965us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:06.771133 (+ 291us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:06.771465 (+ 332us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.771492 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:06.771550 (+ 58us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:06.771606 (+ 56us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.771682 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.771808 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":60657024}
I20260619 21:16:06.814702 2465 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.841053 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.841387 2465 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:06.842015 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 68066.3us
I20260619 21:16:06.842777 1782 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.843768 1859 tablet_bootstrap.cc:654] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:06.826805 2422 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:06.846223 2600 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:06.847929 2421 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:06.856817 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.847434 2621 raft_consensus.cc:2726] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:06.855252 2421 raft_consensus.cc:686] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:06.854086 2424 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:06.861956 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.862171 2453 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.869485 1635 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:06.852954 2425 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.863638 2406 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.864464 2625 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.865207 1741 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:06.852201 2460 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.863521 1860 raft_consensus.cc:348] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:06.882620 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 127260us
I20260619 21:16:06.885283 1860 raft_consensus.cc:374] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:06.885600 1860 raft_consensus.cc:729] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:06.871325 2424 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.879230 1635 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.882400 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 413) took 1346 ms. Trace:
I20260619 21:16:06.886245 1708 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.887100 1860 consensus_queue.cc:260] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:06.890028 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 127867us
I20260619 21:16:06.879108 2453 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.893016 2625 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.893187 2453 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.893666 2453 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:06.896551 1708 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:06.898381 2424 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:06.899209 2425 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.899566 2349 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:06.899989 2349 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:06.900362 2349 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:06.901340 1708 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.901859 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.902609 1741 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:06.887290 1735 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:06.904917 1710 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:06.882555 2527 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:06.888423 2564 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:06.910790 2537 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.909046 2559 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:06.908628 2421 consensus_queue.cc:237] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.908879 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:06.910285 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:06.908859 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:06.913691 1708 leader_election.cc:302] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:06.905448 2625 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:06.915524 2460 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.907347 2406 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.923892 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:06.924926 1735 raft_consensus.cc:2379] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:06.925964 2422 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:06.926663 2422 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:06.927806 2599 consensus_queue.cc:1046] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:06.928740 2581 raft_consensus.cc:3032] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:06.929111 2581 raft_consensus.cc:729] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:06.933495 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:06.952853 1564 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37726 (request call id 97) took 6174 ms. Trace:
I20260619 21:16:06.966209 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 423) took 1660 ms. Trace:
I20260619 21:16:06.913337 2559 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:06.932719 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:06.941293 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:06.985332 1735 raft_consensus.cc:2379] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:06.940034 2537 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:06.986477 2453 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.938512 1563 catalog_manager.cc:5495] T 84c655fb144a4c11b8902e23b4fac063 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:16:06.966675 1738 rpcz_store.cc:276] 0619 21:16:05.305986 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.396117 (+1090131us) service_pool.cc:227] Handling call
0619 21:16:06.396550 (+ 433us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:06.415352 (+ 18802us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.416373 (+ 1021us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:06.801084 (+384711us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b2c00000238. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719d4a1e11 00007f719d4b1442 00007f719eaf9c11 00007f719eabe58a 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772
0619 21:16:06.801678 (+ 594us) log.cc:842] Serialized 26 byte log entry
0619 21:16:06.872702 (+ 71024us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:06.890395 (+ 17693us) log.cc:842] Serialized 14 byte log entry
0619 21:16:06.913297 (+ 22902us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.917081 (+ 3784us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:06.936071 (+ 18990us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719e740bee 00007f719e73ff36 00007f719e73fb58 00007f719e749551 00007f719e749509 00007f719e749499
0619 21:16:06.943252 (+ 7181us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b18002d39a0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb96638 00007f719bb99b11 00007f719bb99ac9 00007f719bb99a59 00007f719bb99a21 00007f719bb98d1d 00007f71a4dfa204 00007f71a4dfa039 00007f719bb78dc6
0619 21:16:06.965951 (+ 22699us) raft_consensus.cc:1671] finished
0619 21:16:06.965984 (+ 33us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.966142 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2286,"prepare.queue_time_us":11178,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":13963,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":3599,"spinlock_wait_cycles":45086336,"thread_start_us":67420,"threads_started":1,"wal-append.queue_time_us":82323}
W20260619 21:16:06.934047 1783 leader_election.cc:341] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:06.947432 2652 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.964648 1635 leader_election.cc:398] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:06.994513 1635 leader_election.cc:401] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260619 21:16:06.920820 2625 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:06.983428 2527 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:06.983326 2661 raft_consensus.cc:2781] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:16:06.968123 2559 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:06.977389 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 69814.3us
I20260619 21:16:06.987223 1858 tablet_bootstrap.cc:492] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:06.938072 1860 ts_tablet_manager.cc:1428] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.123s user 0.005s sys 0.004s
I20260619 21:16:06.889174 1742 rpcz_store.cc:276] 0619 21:16:05.536245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.452984 (+916739us) service_pool.cc:227] Handling call
0619 21:16:06.453466 (+ 482us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:06.459843 (+ 6377us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.459871 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:06.532811 (+ 72940us) log.cc:842] Serialized 26 byte log entry
0619 21:16:06.741156 (+208345us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:06.757882 (+ 16726us) log.cc:842] Serialized 14 byte log entry
0619 21:16:06.882022 (+124140us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:06.882121 (+ 99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:06.882174 (+ 53us) raft_consensus.cc:1671] finished
0619 21:16:06.882188 (+ 14us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:06.882334 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":8813,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":46617,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":234,"thread_start_us":210644,"threads_started":2,"wal-append.queue_time_us":255116}
I20260619 21:16:06.969511 2581 consensus_queue.cc:260] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:06.994437 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 60458.1us
W20260619 21:16:07.029990 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 65804.2us
I20260619 21:16:06.966710 1564 rpcz_store.cc:276] 0619 21:16:00.767730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:00.772396 (+ 4666us) service_pool.cc:227] Handling call
0619 21:16:01.142484 (+370088us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400370ad4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43af0 00007f719eac6f2a 00007f71a4a18d6c 00007f71a4b12c3e 00007f71a2e24fb4 00007f71a2e24f42 00007f71a2e24e71 00007f71a2e24dec 00007f71a2e240a2 00007f719cee894c 00007f719cee7d86
0619 21:16:06.942015 (+5799531us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:16:05.253719 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:16:05.254283 (+ 564us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:16:05.254325 (+ 42us) write_op.cc:435] Acquired schema lock
0619 21:16:05.254347 (+ 22us) tablet.cc:660] Decoding operations
0619 21:16:05.254732 (+ 385us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:16:05.254799 (+ 67us) write_op.cc:641] Partition lock acquired for write op
0619 21:16:05.254826 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0619 21:16:05.255006 (+ 180us) tablet.cc:699] Row locks acquired
0619 21:16:05.255035 (+ 29us) write_op.cc:260] PREPARE: finished
0619 21:16:05.255184 (+ 149us) write_op.cc:270] Start()
0619 21:16:05.255290 (+ 106us) write_op.cc:276] Timestamp: P: 1781903765255137 usec, L: 0
0619 21:16:05.255310 (+ 20us) op_driver.cc:331] REPLICATION: starting
0619 21:16:05.255973 (+ 663us) log.cc:842] Serialized 531 byte log entry
0619 21:16:05.995590 (+739617us) op_driver.cc:444] REPLICATION: finished
0619 21:16:06.205436 (+209846us) write_op.cc:301] APPLY: starting
0619 21:16:06.242918 (+ 37482us) tablet.cc:1365] starting BulkCheckPresence
0619 21:16:06.244361 (+ 1443us) tablet.cc:1368] finished BulkCheckPresence
0619 21:16:06.316274 (+ 71913us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:16:06.316702 (+ 428us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:16:06.343059 (+ 26357us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0619 21:16:06.343134 (+ 75us) write_op.cc:312] APPLY: finished
0619 21:16:06.343659 (+ 525us) log.cc:842] Serialized 22 byte log entry
0619 21:16:06.343952 (+ 293us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:06.344125 (+ 173us) write_op.cc:454] Released schema lock
0619 21:16:06.344200 (+ 75us) write_op.cc:341] FINISH: Updating metrics
0619 21:16:06.344700 (+ 500us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:06.344738 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":170295,"reported_tablets":12,"spinlock_wait_cycles":613488896,"child_traces":[["op",{"apply.queue_time_us":186839,"apply.run_cpu_time_us":3956,"apply.run_wall_time_us":143685,"num_ops":1,"prepare.queue_time_us":162022,"prepare.run_cpu_time_us":21139,"prepare.run_wall_time_us":330108,"raft.queue_time_us":26198,"raft.run_cpu_time_us":16558,"raft.run_wall_time_us":190569,"replication_time_us":629663,"thread_start_us":595359,"threads_started":4,"wal-append.queue_time_us":518817}]]}
I20260619 21:16:07.032912 1858 ts_tablet_manager.cc:1397] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 1.077s user 0.022s sys 0.029s
I20260619 21:16:07.033913 1741 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:16:07.033821 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 128976us
W20260619 21:16:07.041972 1710 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.042510 1710 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.044572 2655 raft_consensus.cc:2726] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:07.048885 1710 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:07.049170 1783 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.049963 2424 raft_consensus.cc:2781] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:07.050463 2424 raft_consensus.cc:686] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.051339 1783 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.051467 2424 consensus_queue.cc:237] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.052332 1783 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.052718 2602 raft_consensus.cc:2781] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:07.053206 2621 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.053277 2602 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:07.053785 2621 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:07.053915 2602 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.054229 2621 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.054860 2453 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.030097 2661 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:07.056016 2661 raft_consensus.cc:3037] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:07.056685 2425 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.057116 2425 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:07.057431 2425 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.059724 1710 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:07.061517 2581 raft_consensus.cc:3037] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.040810 1563 catalog_manager.cc:5495] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:16:06.997531 2625 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.064404 1783 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:06.988521 2349 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:06.997303 2527 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:07.036561 2406 raft_consensus.cc:3037] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.066404 1563 catalog_manager.cc:5495] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:16:07.079214 1635 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:07.082880 1860 tablet_bootstrap.cc:492] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
W20260619 21:16:07.083273 1634 leader_election.cc:334] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.532s (SENT)
W20260619 21:16:07.085160 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 43964.6us
I20260619 21:16:07.069423 2660 raft_consensus.cc:2781] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:07.085774 1634 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.073196 2527 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.086545 1635 leader_election.cc:398] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.087222 1737 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:07.085378 2600 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.097142 2600 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:07.097617 2600 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.098140 2422 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.100158 2422 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.089844 2660 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:07.101132 2599 raft_consensus.cc:2726] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.101481 2660 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.100931 2621 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.102427 1859 tablet_bootstrap.cc:492] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:07.100713 1563 catalog_manager.cc:5495] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:16:07.074188 2606 raft_consensus.cc:2726] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.095256 2551 consensus_queue.cc:1046] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:07.103693 2621 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.095005 2465 consensus_queue.cc:1046] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:07.104007 1563 catalog_manager.cc:5495] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b2e7f02778ec43bdbca03dd013687c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260619 21:16:07.096113 2349 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.073923 2655 consensus_queue.cc:1046] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260619 21:16:07.112317 2655 raft_consensus.cc:3032] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:07.112569 2551 raft_consensus.cc:3032] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:07.112684 2655 raft_consensus.cc:729] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.113070 2551 raft_consensus.cc:729] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.102943 1859 ts_tablet_manager.cc:1397] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.505s user 0.024s sys 0.004s
I20260619 21:16:07.111547 1783 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.091044 1635 leader_election.cc:401] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:07.110469 2620 consensus_queue.cc:1035] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:07.076254 2623 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:07.113833 2655 consensus_queue.cc:260] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.115465 2623 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.116636 2655 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.118136 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.118836 1666 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:07.119607 2652 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.119865 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.121155 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.122644 2537 raft_consensus.cc:3032] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:07.123010 2537 raft_consensus.cc:729] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.123945 2661 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.123827 2537 consensus_queue.cc:260] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.125337 2537 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.126612 1858 raft_consensus.cc:348] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.125373 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.127393 1858 raft_consensus.cc:374] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:07.127660 1858 raft_consensus.cc:729] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:16:07.126122 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.128260 1812 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:07.128306 1858 consensus_queue.cc:260] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.129627 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.129915 2623 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.130455 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.130571 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.131275 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.149102 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.149230 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.149685 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.149952 1812 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 4: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:07.131665 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.150624 1814 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:07.132064 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.131306 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.152014 1815 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:07.126219 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.152885 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.133771 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.153358 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.154963 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.155752 1663 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.157550 1817 raft_consensus.cc:1226] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:07.157328 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.158432 1660 raft_consensus.cc:2379] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:07.158960 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.137234 2628 raft_consensus.cc:2781] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:07.159559 1817 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.159686 2628 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:07.160331 2628 raft_consensus.cc:3037] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.142442 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.143059 1783 leader_election.cc:398] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:07.113945 2551 consensus_queue.cc:260] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.143772 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.150997 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.133009 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.170876 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.136974 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.171819 1664 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.172312 1666 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:07.166214 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.173203 2551 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
W20260619 21:16:07.173147 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 86719.6us
I20260619 21:16:07.173691 1662 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.174521 2453 raft_consensus.cc:3037] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.176016 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.169337 1783 leader_election.cc:401] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:07.179342 1817 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:07.176931 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.193040 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.194474 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.195917 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.189643 1859 raft_consensus.cc:348] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.196784 1859 raft_consensus.cc:374] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:07.197085 1859 raft_consensus.cc:729] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:07.197736 1859 consensus_queue.cc:260] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.200202 1859 ts_tablet_manager.cc:1428] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.086s user 0.006s sys 0.000s
I20260619 21:16:07.201514 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.197096 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.176896 1665 raft_consensus.cc:1226] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.204726 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:16:07.206538 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 65831.1us
I20260619 21:16:07.207762 1859 tablet_bootstrap.cc:492] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:07.214311 2661 leader_election.cc:288] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.225214 2623 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.225620 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.227422 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.232070 1782 leader_election.cc:302] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.232826 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.234047 1665 raft_consensus.cc:2379] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:07.235284 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.235550 2606 raft_consensus.cc:2726] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.242956 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.241127 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.240685 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.245501 1818 raft_consensus.cc:1226] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:07.249342 1860 tablet_bootstrap.cc:654] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:07.184777 2425 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:07.246045 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 111461us
I20260619 21:16:07.254796 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.219779 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 445) took 1455 ms. Trace:
I20260619 21:16:07.236857 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.238480 2406 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.266497 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.289572 2425 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.290736 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.218434 2602 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:07.273195 1634 leader_election.cc:334] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.534s (SENT)
I20260619 21:16:07.273295 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.274021 1810 raft_consensus.cc:1261] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:07.267275 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:07.298110 1669 raft_consensus.cc:1261] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260619 21:16:07.269331 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.299224 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.300118 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.300412 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.300745 1810 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.301608 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.302042 2623 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3.
I20260619 21:16:07.302202 1668 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.303175 2602 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.304373 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.274356 1783 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:07.305572 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.305665 1663 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:07.306174 1783 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.306645 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.306945 1663 raft_consensus.cc:1226] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:07.307096 1783 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.189601 2600 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.274853 1812 raft_consensus.cc:1226] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:07.315507 1858 ts_tablet_manager.cc:1428] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.281s user 0.006s sys 0.002s
I20260619 21:16:07.275063 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.276841 1782 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.277654 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.282152 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.277693 1815 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:07.213060 2660 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.268991 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.269073 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.286480 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.320724 1818 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.287184 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.287448 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.267431 1809 rpcz_store.cc:276] 0619 21:16:05.763898 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:06.225674 (+461776us) service_pool.cc:227] Handling call
0619 21:16:06.226713 (+ 1039us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:06.764483 (+537770us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:06.765922 (+ 1439us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:06.833117 (+ 67195us) log.cc:842] Serialized 26 byte log entry
0619 21:16:07.070729 (+237612us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:16:07.072575 (+ 1846us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:07.076204 (+ 3629us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:07.107548 (+ 31344us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719e740bee 00007f719e73ff36 00007f719e73fb58 00007f719e749551 00007f719e749509 00007f719e749499
0619 21:16:07.144702 (+ 37154us) raft_consensus.cc:1671] finished
0619 21:16:07.144727 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:07.215745 (+ 71018us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1144,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":2939,"spinlock_wait_cycles":22919936,"thread_start_us":232340,"threads_started":1,"wal-append.queue_time_us":263327}
I20260619 21:16:07.287694 2527 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.261924 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.262063 2625 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.309516 1664 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:07.335978 2625 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.296180 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.299230 1669 raft_consensus.cc:1226] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.316500 2600 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.314251 1817 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:07.308363 2453 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.307669 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.337615 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.338084 1811 raft_consensus.cc:2379] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:07.343370 1859 tablet_bootstrap.cc:654] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:07.344388 1816 raft_consensus.cc:1226] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.345381 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.347896 1710 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.318027 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.282637 2628 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.323900 1858 tablet_bootstrap.cc:492] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:07.343389 1818 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:07.331940 1663 raft_consensus.cc:1226] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.336205 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.349368 1815 raft_consensus.cc:2379] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:07.329409 1738 raft_consensus.cc:2454] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:16:07.293452 1662 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:07.363422 1811 raft_consensus.cc:1226] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.340761 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.365734 1736 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:07.357884 1744 raft_consensus.cc:2354] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4.
I20260619 21:16:07.352210 1810 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:07.320922 2623 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:07.367372 2623 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:07.367787 2623 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:16:07.368887 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 115506us
I20260619 21:16:07.369616 1782 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.372360 1782 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.373086 1813 raft_consensus.cc:2379] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:07.373628 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:07.368494 1710 leader_election.cc:341] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.375651 1739 raft_consensus.cc:1226] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:07.376063 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.376992 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.377118 2621 consensus_queue.cc:1046] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:07.316555 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.378618 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.379220 1812 raft_consensus.cc:2379] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:07.379712 2620 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:07.362528 2628 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.383090 1739 raft_consensus.cc:1226] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:07.383497 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.384114 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.384930 1739 raft_consensus.cc:2354] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.385761 1736 raft_consensus.cc:1226] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:16:07.385711 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.386270 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.386835 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.387068 1737 raft_consensus.cc:3037] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.387982 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.388671 1736 raft_consensus.cc:2354] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.389621 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:07.365156 1634 leader_election.cc:334] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.521s (SENT)
I20260619 21:16:07.390604 2527 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.321312 2592 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.391503 1741 raft_consensus.cc:1226] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:07.391433 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.391974 2592 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:07.392508 2592 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.392850 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.393574 1744 raft_consensus.cc:2354] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.367967 2424 raft_consensus.cc:2741] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260619 21:16:07.321137 2606 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:07.400106 2606 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:07.400522 2606 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.393378 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.401831 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.404666 2623 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.407402 2623 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.366977 2559 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.410370 2559 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.411006 1665 raft_consensus.cc:1226] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.348740 2625 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.388007 1710 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:07.413589 1710 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:16:07.414847 1710 leader_election.cc:341] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.414729 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.415472 1710 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.391571 1634 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.356069 1667 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:07.412801 2625 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:07.417292 2564 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.421756 2564 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:07.421840 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.422247 2564 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.361970 1668 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
W20260619 21:16:07.425329 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 53052.6us
W20260619 21:16:07.424337 1710 leader_election.cc:341] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.429630 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.434814 1810 raft_consensus.cc:1226] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:07.423184 2581 raft_consensus.cc:2726] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.424604 2460 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:07.441421 2460 raft_consensus.cc:686] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.442371 2460 consensus_queue.cc:237] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.448468 1634 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.445581 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.423004 2625 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.450414 1743 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:07.424667 2424 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:07.453186 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.454277 1742 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:07.425115 2425 consensus_queue.cc:1046] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260619 21:16:07.458400 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.460426 1743 raft_consensus.cc:2396] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:07.461426 1817 raft_consensus.cc:1226] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.462072 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.463996 1742 raft_consensus.cc:2354] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:07.463629 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.450709 2621 consensus_queue.cc:1225] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer d53146739e3240239c0a3d5ea90e07fe log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260619 21:16:07.465483 1738 raft_consensus.cc:2354] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4.
I20260619 21:16:07.459280 2425 raft_consensus.cc:3032] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:07.466322 2425 raft_consensus.cc:729] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.466552 1739 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:07.467093 2425 consensus_queue.cc:260] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.452324 2581 raft_consensus.cc:2741] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260619 21:16:07.468885 1635 leader_election.cc:341] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.465198 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.469466 2559 consensus_queue.cc:1046] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260619 21:16:07.482471 2592 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.484491 2592 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.485653 2660 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.485801 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.472465 2406 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260619 21:16:07.456513 2527 consensus_queue.cc:1046] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260619 21:16:07.489280 1813 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 4: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:07.491365 2559 raft_consensus.cc:3032] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:07.491753 2559 raft_consensus.cc:729] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.491906 2527 raft_consensus.cc:3032] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:07.492244 2527 raft_consensus.cc:729] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.492588 2559 consensus_queue.cc:260] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.493835 2559 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.494611 2620 raft_consensus.cc:3032] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:07.495141 2620 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.497153 2527 consensus_queue.cc:260] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.498392 2527 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.480325 2606 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.499502 1814 raft_consensus.cc:1226] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:07.502233 2606 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.502933 2606 consensus_queue.cc:1046] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:16:07.504056 2602 consensus_queue.cc:1046] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:07.470216 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.470525 1741 raft_consensus.cc:3037] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:07.473340 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 46416us
I20260619 21:16:07.476832 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:07.472798 2453 consensus_queue.cc:1035] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260619 21:16:07.477911 1739 raft_consensus.cc:1226] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:07.479892 1710 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.480437 2425 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.456650 2424 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:07.482100 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.509119 1710 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.492174 1811 raft_consensus.cc:1226] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:07.513506 1634 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.513837 1743 raft_consensus.cc:2379] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:07.518059 1635 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:07.505606 2602 raft_consensus.cc:3032] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:07.531399 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.532104 1743 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.508548 2424 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.496193 2620 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.523715 2602 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.535147 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.516301 2551 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:07.537211 2581 raft_consensus.cc:2781] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:07.546911 2628 consensus_queue.cc:1046] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:16:07.547377 2581 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:07.547786 2581 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.538532 1739 raft_consensus.cc:1261] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260619 21:16:07.554188 1739 pending_rounds.cc:76] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Aborting all ops after (but not including) 2
I20260619 21:16:07.555126 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.556496 1668 raft_consensus.cc:1226] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:07.516451 2660 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:07.558200 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.564445 1742 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:07.559342 1635 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:07.561589 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.560225 2551 raft_consensus.cc:686] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.554527 1739 pending_rounds.cc:97] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260619 21:16:07.567539 1739 raft_consensus.cc:2866] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:07.522930 1737 raft_consensus.cc:2454] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:07.572638 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.573477 2551 consensus_queue.cc:237] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.574796 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.571563 2620 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.577517 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.578135 1665 raft_consensus.cc:3037] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:07.580834 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.581158 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.565824 2600 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:07.581168 2602 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.524065 2406 consensus_queue.cc:1046] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260619 21:16:07.583401 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.569906 2660 raft_consensus.cc:686] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:07.582986 1740 raft_consensus.cc:1226] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.586208 2660 consensus_queue.cc:237] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.515936 2564 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.583449 2602 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.606954 1740 raft_consensus.cc:1226] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.571557 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:07.631289 2581 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.633162 1742 raft_consensus.cc:1226] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:07.591316 1783 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:07.597076 2623 raft_consensus.cc:3032] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:07.635190 2623 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.601540 2592 raft_consensus.cc:3032] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:07.594314 2261 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:07.594672 2600 raft_consensus.cc:686] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.595335 2406 raft_consensus.cc:3032] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:07.610194 2453 consensus_queue.cc:1046] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:07.645514 1858 tablet_bootstrap.cc:654] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:07.645414 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.645725 2406 raft_consensus.cc:729] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.646132 1811 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
W20260619 21:16:07.612414 1635 consensus_peers.cc:487] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:16:07.614742 1661 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:07.628255 1859 tablet_bootstrap.cc:492] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:07.647938 1736 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:07.642429 2592 raft_consensus.cc:729] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.644121 2261 raft_consensus.cc:686] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
W20260619 21:16:07.629873 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 46233.6us
I20260619 21:16:07.644692 2621 raft_consensus.cc:2781] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:07.645812 2556 raft_consensus.cc:3032] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:07.598544 2564 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.653887 2453 consensus_queue.cc:1046] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:07.646698 2406 consensus_queue.cc:260] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.654935 2585 raft_consensus.cc:3032] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:07.655303 2585 raft_consensus.cc:729] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.655519 2406 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.636096 2623 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.655766 2581 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.656370 2623 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.656140 2585 consensus_queue.cc:260] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.649029 1859 ts_tablet_manager.cc:1397] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.448s user 0.019s sys 0.000s
W20260619 21:16:07.644158 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 52922us
I20260619 21:16:07.642545 1743 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:07.649842 2592 consensus_queue.cc:260] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.651643 2621 raft_consensus.cc:686] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.650146 2261 consensus_queue.cc:237] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.652055 2600 consensus_queue.cc:237] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.665576 2421 consensus_queue.cc:1046] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:07.665874 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.667053 1783 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:07.673890 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 211798us
I20260619 21:16:07.675315 1782 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:07.679633 1708 leader_election.cc:341] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.681691 1667 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:07.680248 1708 leader_election.cc:302] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.652927 2556 raft_consensus.cc:729] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.666743 2621 consensus_queue.cc:237] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.689895 1708 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.694263 1634 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.697757 2421 raft_consensus.cc:3032] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:07.698189 2421 raft_consensus.cc:729] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.680644 2606 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:07.683383 1737 raft_consensus.cc:1226] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:07.705962 1708 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.684816 2673 raft_consensus.cc:2726] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.686090 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:07.710731 1634 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.690770 2655 raft_consensus.cc:2726] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:07.717559 1708 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.704686 2585 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:07.707458 2460 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:07.724568 2655 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.725082 1708 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.729097 2592 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:07.714502 2625 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.731081 1859 raft_consensus.cc:348] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.737274 1859 raft_consensus.cc:374] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:07.737670 1859 raft_consensus.cc:729] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:07.735273 1708 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:07.741139 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 73079us
I20260619 21:16:07.716576 2606 raft_consensus.cc:686] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:07.749953 1708 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.745152 1859 consensus_queue.cc:260] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.724064 2551 raft_consensus.cc:2726] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.753914 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:07.726533 2581 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:07.700647 2421 consensus_queue.cc:260] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.739213 2425 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
W20260619 21:16:07.757687 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 47036.2us
I20260619 21:16:07.761467 1860 tablet_bootstrap.cc:492] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:07.761987 1860 ts_tablet_manager.cc:1397] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.686s user 0.023s sys 0.003s
I20260619 21:16:07.753156 2652 raft_consensus.cc:2726] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.763779 1735 raft_consensus.cc:2379] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 4: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:16:07.763221 1708 leader_election.cc:341] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.794848 1708 leader_election.cc:302] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.758711 2655 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:07.804865 1859 ts_tablet_manager.cc:1428] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.140s user 0.013s sys 0.000s
I20260619 21:16:07.809448 1634 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.761085 2581 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:07.813871 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 50678.9us
I20260619 21:16:07.815660 1708 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.722702 2556 consensus_queue.cc:260] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.824383 2683 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.734071 2685 consensus_queue.cc:1046] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260619 21:16:07.830667 1782 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:07.831559 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 64210.6us
I20260619 21:16:07.821303 2652 raft_consensus.cc:2726] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:07.757619 2684 consensus_queue.cc:1046] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260619 21:16:07.853541 1782 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.828975 2683 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:07.855567 1708 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.858359 1708 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.862221 1782 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.863783 1708 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.865516 1708 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.822613 2581 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.866403 1782 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.864018 1860 raft_consensus.cc:348] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.869608 1708 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:07.869992 1860 raft_consensus.cc:374] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:07.870393 1860 raft_consensus.cc:729] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:07.873301 1782 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.850006 2421 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.853690 2673 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:07.871243 1860 consensus_queue.cc:260] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.881954 1634 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:07.861060 2527 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:07.830304 2606 consensus_queue.cc:237] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.862768 2676 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:07.890094 1858 tablet_bootstrap.cc:492] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:07.890707 1858 ts_tablet_manager.cc:1397] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.574s user 0.026s sys 0.013s
I20260619 21:16:07.863363 2652 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260619 21:16:07.883214 2686 log.cc:924] Time spent T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20260619 21:16:07.865638 2655 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:07.867238 2465 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:07.873136 2553 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:07.916163 2424 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.918888 2424 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:07.886737 2551 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:07.922004 2460 consensus_queue.cc:1046] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:07.922812 2556 raft_consensus.cc:3037] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:07.923162 1634 leader_election.cc:398] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4.
I20260619 21:16:07.923552 2689 raft_consensus.cc:3032] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:07.925516 2683 raft_consensus.cc:3032] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:07.926862 1665 raft_consensus.cc:2454] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:07.880362 1738 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:07.928593 2676 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:07.891230 2625 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:07.886058 2559 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:07.867270 2620 consensus_queue.cc:1046] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260619 21:16:07.971985 2620 raft_consensus.cc:3032] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:07.939462 1858 raft_consensus.cc:348] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:07.982311 1858 raft_consensus.cc:374] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:07.982693 1858 raft_consensus.cc:729] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
W20260619 21:16:07.942046 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 62197.2us
I20260619 21:16:07.922004 1810 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:07.921620 2673 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260619 21:16:07.962899 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 89641.5us
I20260619 21:16:07.933151 2602 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:07.963943 2689 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.965615 2683 raft_consensus.cc:729] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:07.886474 2527 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:07.970723 1815 raft_consensus.cc:1226] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260619 21:16:07.973912 1859 tablet_bootstrap.cc:492] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:07.977962 2581 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:07.978576 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 97784.7us
W20260619 21:16:07.970072 2677 log.cc:924] Time spent T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.141s user 0.000s sys 0.001s
I20260619 21:16:07.981223 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:07.986467 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:08.007561 1814 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:07.983525 1858 consensus_queue.cc:260] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:07.997588 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 74548.7us
I20260619 21:16:07.966183 2551 raft_consensus.cc:686] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.010810 1708 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.011914 1634 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:08.013888 1635 leader_election.cc:341] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:07.988060 2620 raft_consensus.cc:729] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:08.016215 1858 ts_tablet_manager.cc:1428] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.125s user 0.003s sys 0.005s
I20260619 21:16:08.017004 1782 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.018118 1782 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.021950 1860 ts_tablet_manager.cc:1428] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.260s user 0.015s sys 0.028s
I20260619 21:16:08.003942 2527 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:08.025413 1858 tablet_bootstrap.cc:492] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:08.028749 1708 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.031925 1782 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.032284 1708 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.022642 2660 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:08.022697 2602 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:08.036757 1634 leader_election.cc:398] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:08.025466 2465 raft_consensus.cc:3032] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:08.041059 1708 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.039458 1634 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.063933 1634 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.064034 1708 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.045395 1782 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.040390 2655 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:08.063293 2694 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:08.040017 2697 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:08.043920 2676 raft_consensus.cc:2781] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:08.074569 2699 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:08.040231 2465 raft_consensus.cc:729] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.044126 2698 raft_consensus.cc:2726] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:08.039922 2564 consensus_queue.cc:1046] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260619 21:16:08.042142 2621 consensus_queue.cc:1046] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260619 21:16:08.031023 2683 consensus_queue.cc:260] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.097673 1634 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.068457 2660 raft_consensus.cc:686] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.108846 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.109880 2465 consensus_queue.cc:260] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.110752 2695 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:08.111888 2703 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:08.101361 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.113703 2676 raft_consensus.cc:686] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:08.114145 1660 raft_consensus.cc:2379] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:08.114372 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.114998 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.123482 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.112429 2703 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.124647 1660 raft_consensus.cc:1226] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:08.077766 2655 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:08.125619 2703 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:08.111920 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.126014 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.077396 2628 raft_consensus.cc:2781] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:08.112787 2465 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:08.126856 1665 raft_consensus.cc:2362] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3. Re-sending same reply.
W20260619 21:16:08.118503 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 74565.5us
I20260619 21:16:08.074581 2693 raft_consensus.cc:2726] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:08.114641 2676 consensus_queue.cc:237] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.125844 2655 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:08.108340 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 44381.3us
I20260619 21:16:08.085701 2553 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:08.117276 1669 raft_consensus.cc:1261] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 1. (term mismatch)
I20260619 21:16:08.136164 1669 pending_rounds.cc:76] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Aborting all ops after (but not including) 0
I20260619 21:16:08.136409 1669 pending_rounds.cc:97] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20260619 21:16:08.136663 1669 raft_consensus.cc:2866] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:08.085819 2600 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:08.032466 2689 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.134514 1708 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.096469 2697 raft_consensus.cc:686] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.042666 2620 consensus_queue.cc:260] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.137302 1634 leader_election.cc:398] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:08.144819 1634 leader_election.cc:401] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260619 21:16:08.138674 1814 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:08.146258 1708 leader_election.cc:302] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.143193 2708 consensus_queue.cc:1046] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:08.147670 2708 raft_consensus.cc:2726] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:08.150765 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.154101 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.155081 1669 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:08.147560 2697 consensus_queue.cc:237] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.158928 1710 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:08.160646 2620 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:08.157411 2564 consensus_queue.cc:1046] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260619 21:16:08.130987 2672 raft_consensus.cc:482] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:08.074066 2551 consensus_queue.cc:237] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.168107 1634 leader_election.cc:398] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:08.171244 1634 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.106195 2261 consensus_queue.cc:1046] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260619 21:16:08.140367 2559 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:08.136189 2628 raft_consensus.cc:686] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:16:08.185137 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 96322.2us
I20260619 21:16:08.164379 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.143543 2600 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.167979 2699 raft_consensus.cc:2781] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:08.170266 2693 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:08.190030 2599 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
W20260619 21:16:08.195417 2623 consensus_meta.cc:287] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.539s user 0.014s sys 0.000s
I20260619 21:16:08.206043 2602 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:08.208088 2701 log.cc:924] Time spent T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260619 21:16:08.208638 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:08.209481 2696 log.cc:924] Time spent T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.136s user 0.001s sys 0.000s
I20260619 21:16:08.207212 1782 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.214545 1634 leader_election.cc:398] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:08.171885 2672 raft_consensus.cc:504] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.206362 2660 consensus_queue.cc:237] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.263818 1667 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:08.265311 1634 leader_election.cc:398] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4.
I20260619 21:16:08.267633 1859 tablet_bootstrap.cc:654] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
W20260619 21:16:08.267979 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 52943.9us
I20260619 21:16:08.269856 1634 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.272006 2623 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:08.276989 1634 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:08.278887 1634 leader_election.cc:341] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:08.279614 1634 leader_election.cc:302] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:08.271075 2406 consensus_meta.cc:287] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.612s user 0.010s sys 0.012s
I20260619 21:16:08.274242 2421 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:08.208104 2602 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.227929 2628 consensus_queue.cc:237] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:08.270641 1811 consensus_meta.cc:287] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.624s user 0.016s sys 0.006s
I20260619 21:16:08.280857 2421 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.285897 1860 tablet_bootstrap.cc:492] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:08.286885 2421 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.282399 2684 raft_consensus.cc:2726] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:08.291723 2709 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election won for term 4
W20260619 21:16:08.277079 2425 consensus_meta.cc:287] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.516s user 0.018s sys 0.005s
I20260619 21:16:08.300607 1669 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:08.292249 2709 raft_consensus.cc:686] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.267853 2599 raft_consensus.cc:686] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:08.307135 1858 tablet_bootstrap.cc:654] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:08.329713 1634 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:08.313613 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 106470us
I20260619 21:16:08.317286 2585 consensus_queue.cc:1035] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:08.318732 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:08.320183 1735 raft_consensus.cc:1261] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260619 21:16:08.279542 2693 raft_consensus.cc:2781] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:08.281819 2672 leader_election.cc:288] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:08.304026 1811 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:08.313438 2425 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260619 21:16:08.326234 2705 log.cc:924] Time spent T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.200s user 0.002s sys 0.000s
I20260619 21:16:08.308617 2688 raft_consensus.cc:2726] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:08.323213 1663 raft_consensus.cc:2454] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:08.341717 2421 raft_consensus.cc:2726] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:08.343382 1665 raft_consensus.cc:1261] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:08.366000 1782 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.372752 1814 raft_consensus.cc:1261] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:08.334708 2655 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.374722 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:08.353536 2693 raft_consensus.cc:686] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:08.354360 1813 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:08.364037 2600 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.378674 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:08.396009 1811 raft_consensus.cc:1261] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:08.396696 1710 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:08.398836 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:08.404078 2551 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:08.404646 2551 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.407662 2551 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.378782 2698 consensus_queue.cc:1035] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260619 21:16:08.411056 2465 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:08.341094 2556 consensus_queue.cc:1035] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260619 21:16:08.361006 2685 consensus_queue.cc:1046] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260619 21:16:08.379397 2672 raft_consensus.cc:2726] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:08.378357 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:08.394315 1744 consensus_meta.cc:287] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.649s user 0.010s sys 0.002s
I20260619 21:16:08.411844 2465 raft_consensus.cc:686] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.427443 1783 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.363775 2695 consensus_queue.cc:1046] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260619 21:16:08.431945 2708 consensus_queue.cc:1035] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260619 21:16:08.436731 2559 raft_consensus.cc:3032] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:08.437376 2559 raft_consensus.cc:729] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:08.438725 2708 consensus_queue.cc:1035] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:08.438659 2559 consensus_queue.cc:260] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.435453 2716 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:08.442973 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:08.443938 1817 raft_consensus.cc:1261] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:08.413328 2710 consensus_queue.cc:1046] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260619 21:16:08.416131 2600 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:08.415916 1661 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:08.441501 2716 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.438659 2625 raft_consensus.cc:3032] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:08.442219 2689 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:08.444283 2672 raft_consensus.cc:2726] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:08.431366 1744 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:08.425750 2711 consensus_queue.cc:1035] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260619 21:16:08.457621 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.454708 2600 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:16:08.477614 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 119080us
I20260619 21:16:08.468099 2602 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:08.483014 2716 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.480983 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.490598 1634 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:08.477603 2699 raft_consensus.cc:686] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.469091 2625 raft_consensus.cc:729] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:08.392542 2709 consensus_queue.cc:237] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.493269 1634 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.496068 2421 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:08.497340 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:08.500912 1782 leader_election.cc:341] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:08.463815 2708 consensus_queue.cc:1035] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260619 21:16:08.484334 2600 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:08.436269 2465 consensus_queue.cc:237] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.431774 2599 consensus_queue.cc:237] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.490231 2453 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:08.542110 1859 tablet_bootstrap.cc:492] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:08.542882 1859 ts_tablet_manager.cc:1397] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 0.569s user 0.012s sys 0.007s
I20260619 21:16:08.494395 1809 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:08.494208 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:08.509502 2406 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:08.502951 2623 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:08.505625 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:08.500018 2625 consensus_queue.cc:260] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.490043 2581 consensus_queue.cc:1035] T 0dea6228e49c44b9889aa709158ef557 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.054s
I20260619 21:16:08.598706 1859 raft_consensus.cc:348] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.601768 1859 raft_consensus.cc:374] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:08.602293 1859 raft_consensus.cc:729] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
W20260619 21:16:08.549345 2527 consensus_meta.cc:287] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.513s user 0.015s sys 0.000s
I20260619 21:16:08.546329 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:08.538338 2421 raft_consensus.cc:686] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:08.608635 2527 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.622215 2702 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:08.623379 1738 raft_consensus.cc:2379] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:08.475701 2693 consensus_queue.cc:237] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:08.624223 1710 leader_election.cc:341] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:08.625063 1740 raft_consensus.cc:1226] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:08.623524 2527 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.623086 2702 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.636592 1736 raft_consensus.cc:1261] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:08.638608 1735 raft_consensus.cc:1261] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:08.640671 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:08.644683 1860 tablet_bootstrap.cc:654] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:08.644683 1859 consensus_queue.cc:260] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.655021 1783 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.656397 2676 consensus_queue.cc:1035] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260619 21:16:08.655066 2698 consensus_queue.cc:1035] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260619 21:16:08.669337 2623 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:08.687546 1859 ts_tablet_manager.cc:1428] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.144s user 0.010s sys 0.014s
I20260619 21:16:08.632598 2699 consensus_queue.cc:237] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.696763 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:08.711994 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.680022 2676 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:08.713621 2676 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.721060 1859 tablet_bootstrap.cc:492] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:08.721167 1858 tablet_bootstrap.cc:492] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:08.721851 1858 ts_tablet_manager.cc:1397] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.701s user 0.032s sys 0.033s
W20260619 21:16:08.723064 1782 leader_election.cc:341] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:08.724911 1783 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.739133 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:08.709874 2702 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:08.728495 2676 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:08.748468 2730 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:08.751008 1858 raft_consensus.cc:348] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.752027 1858 raft_consensus.cc:374] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:08.752372 1858 raft_consensus.cc:729] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
I20260619 21:16:08.761461 1661 raft_consensus.cc:1261] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:08.765010 1668 raft_consensus.cc:2396] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260619 21:16:08.685105 2421 consensus_queue.cc:237] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.744110 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:08.779983 2700 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:08.780478 1736 raft_consensus.cc:2379] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:08.797458 2698 consensus_queue.cc:1035] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:08.799198 1858 consensus_queue.cc:260] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.803274 2732 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:08.804432 2703 raft_consensus.cc:3032] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:08.805084 2703 raft_consensus.cc:729] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:08.806025 1736 raft_consensus.cc:1261] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:08.750626 2730 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.809728 2702 consensus_queue.cc:1035] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:08.826900 1858 ts_tablet_manager.cc:1428] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.105s user 0.008s sys 0.003s
I20260619 21:16:08.804373 2732 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.846004 2730 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.847574 2732 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:08.848369 2700 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:08.849462 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.806383 2703 consensus_queue.cc:260] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.824124 2684 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:08.870904 1858 tablet_bootstrap.cc:492] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:08.906385 1782 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:08.878979 1634 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 2.085s (SENT) [suppressed 30 similar messages]
I20260619 21:16:08.890707 1736 raft_consensus.cc:1261] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:08.907828 2703 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:08.924284 1664 raft_consensus.cc:1261] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:08.869589 2684 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.935307 1817 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
W20260619 21:16:08.936810 2733 log.cc:924] Time spent T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.117s user 0.000s sys 0.000s
I20260619 21:16:08.937005 2424 raft_consensus.cc:3032] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:08.938313 1662 raft_consensus.cc:1261] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260619 21:16:08.941319 1859 tablet_bootstrap.cc:654] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:08.862954 2700 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.915640 2628 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:08.955595 1668 raft_consensus.cc:1261] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:08.948909 1662 pending_rounds.cc:76] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Aborting all ops after (but not including) 2
I20260619 21:16:08.968178 1858 tablet_bootstrap.cc:654] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:08.949672 2424 raft_consensus.cc:729] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:08.981992 2606 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:08.982921 2606 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:08.987607 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:08.987942 2606 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:08.989740 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:08.991118 1736 raft_consensus.cc:2379] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:08.994639 1782 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:08.960139 1662 pending_rounds.cc:97] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260619 21:16:08.998726 2606 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:08.958662 2702 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:09.003818 2726 consensus_queue.cc:1046] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260619 21:16:09.004165 2424 consensus_queue.cc:260] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.007469 2424 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:09.007367 2712 consensus_queue.cc:1035] T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260619 21:16:08.998338 1662 raft_consensus.cc:2866] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:09.012454 2713 consensus_queue.cc:1035] T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260619 21:16:08.974674 2422 raft_consensus.cc:3032] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:09.017015 2422 raft_consensus.cc:729] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:09.014894 2702 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.034036 2421 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.101255 1860 tablet_bootstrap.cc:492] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:09.103148 2708 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:09.015699 2698 consensus_queue.cc:1035] T a3c98fc9dacf48e8994724fe2e303166 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
I20260619 21:16:09.107218 1860 ts_tablet_manager.cc:1397] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.822s user 0.037s sys 0.037s
I20260619 21:16:09.308722 2620 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:09.306078 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 296114us
I20260619 21:16:09.565876 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.566802 1742 raft_consensus.cc:2379] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:09.567343 1817 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:09.249550 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.569671 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.573043 1739 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:09.170653 1743 raft_consensus.cc:2454] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:16:09.577001 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 467) took 1195 ms. Trace:
I20260619 21:16:09.073004 2261 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:09.591404 1783 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:09.096063 2660 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:09.107033 2740 log.cc:924] Time spent T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.201s user 0.002s sys 0.000s
W20260619 21:16:09.204653 2729 log.cc:924] Time spent T 0d45d883f7fa4dd285976666f28c0770 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.559s user 0.000s sys 0.005s
W20260619 21:16:09.213217 2623 consensus_meta.cc:287] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.503s user 0.013s sys 0.003s
W20260619 21:16:09.156220 2747 log.cc:924] Time spent T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260619 21:16:09.596835 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:09.155675 2600 consensus_meta.cc:287] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.609s user 0.010s sys 0.006s
W20260619 21:16:09.390853 2743 log.cc:924] Time spent T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.202s user 0.000s sys 0.001s
I20260619 21:16:09.097874 2693 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:09.585351 1743 rpcz_store.cc:276] 0619 21:16:08.382032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.382140 (+ 108us) service_pool.cc:227] Handling call
0619 21:16:08.739001 (+356861us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:09.170336 (+431335us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54001a28a8. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a37434 00007f71a3a00437 00007f71a39ffbc8 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86
0619 21:16:09.576910 (+406574us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":28272256}
W20260619 21:16:09.205415 2746 log.cc:924] Time spent T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.101s user 0.000s sys 0.001s
I20260619 21:16:09.484025 2726 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.620103 2726 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.622687 2726 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.622764 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.174330 2684 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.155067 2683 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:09.304795 2556 raft_consensus.cc:3032] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:09.206427 2606 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:09.168044 2702 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.587841 2620 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.169165 2672 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.571676 2559 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:09.131625 2689 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.210747 2652 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:09.152184 2700 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.593876 2673 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:09.617125 1783 leader_election.cc:302] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:09.631788 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.610128 2623 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:09.599735 2261 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.391144 2736 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.108022 2421 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.587914 2708 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.616626 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.630244 2556 raft_consensus.cc:729] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:09.600450 2660 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.614604 2600 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:09.652740 1635 leader_election.cc:341] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260619 21:16:09.641119 1809 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.569s user 0.017s sys 0.035s
I20260619 21:16:09.150898 2422 consensus_queue.cc:260] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.625510 2693 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.667517 1818 raft_consensus.cc:1261] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:09.669423 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.641167 2698 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:09.648448 2700 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.655004 2759 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:09.669317 1809 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.630s user 0.018s sys 0.035s
I20260619 21:16:09.656606 2623 raft_consensus.cc:2781] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:16:09.640269 2652 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.640430 2672 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.640420 2689 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.690238 2652 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.691128 2672 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.692683 2261 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.693534 1860 raft_consensus.cc:348] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.694262 1860 raft_consensus.cc:374] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:09.694504 1860 raft_consensus.cc:729] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:09.695127 1860 consensus_queue.cc:260] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:09.692701 1744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 457) took 2242 ms (client timeout 2085 ms). Trace:
W20260619 21:16:09.696096 1744 rpcz_store.cc:269] 0619 21:16:07.450087 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:07.451047 (+ 960us) service_pool.cc:227] Handling call
0619 21:16:09.692421 (+2241374us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54002d55d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:09.692572 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":573412224}
I20260619 21:16:09.696741 1860 ts_tablet_manager.cc:1428] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.132s user 0.002s sys 0.000s
W20260619 21:16:09.684795 1783 leader_election.cc:341] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:09.683843 1662 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:09.708667 1783 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:09.709462 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.710210 1662 raft_consensus.cc:2454] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:16:09.711162 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:09.711617 2757 raft_consensus.cc:2726] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:09.713624 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.713891 1860 tablet_bootstrap.cc:492] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:09.714002 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.714589 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:09.716568 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:09.677256 2585 consensus_queue.cc:1035] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260619 21:16:09.675596 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 510) took 1023 ms. Trace:
I20260619 21:16:09.664812 2730 consensus_queue.cc:1046] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260619 21:16:09.687964 1858 tablet_bootstrap.cc:492] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe: No bootstrap required, opened a new log
I20260619 21:16:09.727080 1858 ts_tablet_manager.cc:1397] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe: Time spent bootstrapping tablet: real 0.865s user 0.040s sys 0.023s
I20260619 21:16:09.664516 2736 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.683964 2759 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.698074 2600 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:09.698360 1667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 415) took 1598 ms (client timeout 1687 ms). Trace:
I20260619 21:16:09.699558 1816 raft_consensus.cc:1261] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:09.699635 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.700474 2708 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:09.728223 1667 rpcz_store.cc:269] 0619 21:16:08.099569 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.100443 (+ 874us) service_pool.cc:227] Handling call
0619 21:16:09.698174 (+1597731us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b54000569d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:09.698277 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":171798272}
I20260619 21:16:09.684496 2556 consensus_queue.cc:260] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.734472 2652 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:09.690587 2689 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.734895 2759 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.735080 2556 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:09.734992 2652 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.736522 1818 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:09.701957 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:09.703290 1661 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:09.683924 2623 raft_consensus.cc:482] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:09.742161 2623 raft_consensus.cc:3037] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:09.742914 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.743146 2730 raft_consensus.cc:3032] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:09.743562 2730 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:09.682610 2700 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.705771 2620 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.705986 2421 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.738112 2765 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:16:09.707451 2628 consensus_meta.cc:287] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.755s user 0.006s sys 0.007s
I20260619 21:16:09.746588 2628 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260619 21:16:09.704712 2727 log.cc:924] Time spent T ffcc16880ccb4a85a786728439b64cc0 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 1.238s user 0.009s sys 0.020s
I20260619 21:16:09.707815 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.674574 2698 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:09.666051 2713 consensus_queue.cc:1035] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260619 21:16:09.710292 1664 raft_consensus.cc:1261] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:09.652963 2673 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:09.683131 2703 consensus_meta.cc:287] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.743s user 0.008s sys 0.007s
I20260619 21:16:09.732275 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.732676 1668 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:09.734025 2564 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
F20260619 21:16:09.734141 396 test_workload.cc:378] Timed out: Timed out waiting for Table Creation
*** Check failure stack trace: ***
I20260619 21:16:09.739462 1817 raft_consensus.cc:1261] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:09.851325 2693 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.851750 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.852036 1817 raft_consensus.cc:1261] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:09.852442 1736 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:09.739948 1669 raft_consensus.cc:1226] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:09.853428 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 432) took 1421 ms. Trace:
*** Aborted at 1781903769 (unix time) try "date -d @1781903769" if you are using GNU date ***
I20260619 21:16:09.853772 1669 rpcz_store.cc:276] 0619 21:16:08.433916 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.442937 (+ 9021us) service_pool.cc:227] Handling call
0619 21:16:09.739574 (+1296637us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:09.739778 (+ 204us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:09.853126 (+113348us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:09.853346 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11958656}
I20260619 21:16:09.740957 2736 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.724884 1810 rpcz_store.cc:276] 0619 21:16:08.650390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.652229 (+ 1839us) service_pool.cc:227] Handling call
0619 21:16:08.664121 (+ 11892us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:08.685562 (+ 21441us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:08.687125 (+ 1563us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:08.736763 (+ 49638us) log.cc:842] Serialized 26 byte log entry
0619 21:16:08.821853 (+ 85090us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:08.821962 (+ 109us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:08.822097 (+ 135us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:09.664798 (+842701us) raft_consensus.cc:1671] finished
0619 21:16:09.664843 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:09.673542 (+ 8699us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6844,"raft.queue_time_us":8610,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":9424,"thread_start_us":78062,"threads_started":1,"wal-append.queue_time_us":141151}
I20260619 21:16:09.735023 1665 raft_consensus.cc:1261] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:09.855990 1666 raft_consensus.cc:2396] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:09.734508 1667 raft_consensus.cc:1261] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:09.748077 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.744349 2730 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.856783 1663 raft_consensus.cc:2454] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:16:09.857149 2730 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:09.853788 1783 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:09.858831 2702 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:09.859189 2702 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:09.859555 2702 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:09.763559 1858 raft_consensus.cc:348] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.860371 1858 raft_consensus.cc:374] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:09.860680 1858 raft_consensus.cc:729] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Initialized, Role: FOLLOWER
W20260619 21:16:09.857961 1634 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.949s user 0.017s sys 0.015s
W20260619 21:16:09.861573 1634 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.952s user 0.018s sys 0.015s
W20260619 21:16:09.861872 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 982745us
I20260619 21:16:09.746477 2765 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.769647 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.782614 1814 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:09.752873 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.765107 2771 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:09.786521 1635 leader_election.cc:341] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:09.786770 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 500) took 1328 ms. Trace:
W20260619 21:16:09.868145 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 81658.5us
I20260619 21:16:09.864928 2460 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.870982 1809 rpcz_store.cc:276] 0619 21:16:08.440868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.448094 (+ 7226us) service_pool.cc:227] Handling call
0619 21:16:08.485076 (+ 36982us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:08.801686 (+316610us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:08.808812 (+ 7126us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:08.853562 (+ 44750us) log.cc:842] Serialized 26 byte log entry
0619 21:16:08.980762 (+127200us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:09.018656 (+ 37894us) log.cc:842] Serialized 14 byte log entry
0619 21:16:09.681720 (+663064us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:09.681812 (+ 92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:09.695508 (+ 13696us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b2400410fa8. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719b9eecf4 00007f719b9ecdfb 00007f719ba7b559 00007f719ba81495 00007f719ba81362 00007f719ba813ee 00007f719b4bf5db 00007f71a39e4dda 00007f719ba812fc 00007f719ba739bd
0619 21:16:09.738902 (+ 43394us) raft_consensus.cc:1671] finished
0619 21:16:09.752295 (+ 13393us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:09.767019 (+ 14724us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7484,"prepare.queue_time_us":34138,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":123662,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":21504,"spinlock_wait_cycles":18954496,"thread_start_us":772063,"threads_started":2,"wal-append.queue_time_us":175403}
I20260619 21:16:09.872907 2460 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.769742 2703 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.874210 2703 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.783658 1813 raft_consensus.cc:1261] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:09.808528 2708 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.877820 2708 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.879674 2460 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.879801 2708 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.865428 2684 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.881158 2684 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.848430 2652 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.846719 2660 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.867476 2585 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.867301 2771 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.883134 2585 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.884150 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.767448 2688 consensus_queue.cc:1035] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260619 21:16:09.884529 2771 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.866370 2765 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.861882 2716 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.806680 2421 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:09.886081 2716 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.886307 2421 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.888013 2716 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.888211 1860 tablet_bootstrap.cc:654] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:09.842772 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.888823 1744 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:09.849622 2700 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.861410 1858 consensus_queue.cc:260] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.749444 2698 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:09.889845 2767 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.842957 2673 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:16:09.838282 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.888132 2421 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.767637 2564 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:09.848748 2769 log.cc:924] Time spent T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.076s user 0.000s sys 0.002s
W20260619 21:16:09.890776 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 243200us
I20260619 21:16:09.892427 1783 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:09.899305 1858 ts_tablet_manager.cc:1428] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe: Time spent starting tablet: real 0.172s user 0.004s sys 0.004s
I20260619 21:16:09.899356 1782 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:09.893975 1635 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:09.890153 2628 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.901744 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.876806 2703 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.906394 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.907159 1744 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:09.909596 2621 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:09.910017 2621 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:09.911840 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:09.917047 1783 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:09.921542 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.918468 1782 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:09.925199 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.925805 1663 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:09.897230 2688 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.901392 2683 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:09.905006 2673 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.910439 2621 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:09.941586 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.891057 2623 raft_consensus.cc:504] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.910567 2759 raft_consensus.cc:2781] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:09.915759 2421 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.947185 2421 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.948578 2564 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:09.948755 1666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 418) took 1845 ms (client timeout 2059 ms). Trace:
I20260619 21:16:09.928336 2453 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election won for term 5
W20260619 21:16:09.949254 1666 rpcz_store.cc:269] 0619 21:16:08.103290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.103848 (+ 558us) service_pool.cc:227] Handling call
0619 21:16:08.491905 (+388057us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:09.948530 (+1456625us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540034fb54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:09.948692 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2944596224}
I20260619 21:16:09.890300 2767 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.950044 2453 raft_consensus.cc:686] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 5 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:09.950403 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 470) took 1567 ms. Trace:
I20260619 21:16:09.950778 1740 rpcz_store.cc:276] 0619 21:16:08.382977 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.384196 (+ 1219us) service_pool.cc:227] Handling call
0619 21:16:08.384426 (+ 230us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:08.633172 (+248746us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:09.933904 (+1300732us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54002b8754. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:09.950298 (+ 16394us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":552739072}
I20260619 21:16:09.951831 2767 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:09.904786 2628 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.951323 2453 consensus_queue.cc:237] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.934365 2700 raft_consensus.cc:2781] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:09.953825 2700 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:09.946266 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.954299 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.954411 2673 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.940747 2688 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.955415 1665 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:09.957114 2759 raft_consensus.cc:686] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:09.957360 1815 raft_consensus.cc:1261] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:09.954313 2700 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:09.958500 2759 consensus_queue.cc:237] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.915570 2693 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:09.920657 2600 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:09.975509 1859 tablet_bootstrap.cc:492] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:09.945606 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.976087 1859 ts_tablet_manager.cc:1397] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 1.267s user 0.032s sys 0.013s
I20260619 21:16:09.946347 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:09.975498 2600 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:09.944675 1669 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:09.977625 2600 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:09.965488 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.978072 1809 raft_consensus.cc:1261] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:09.978973 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.970045 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.954365 2683 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:09.981184 1735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 477) took 1312 ms. Trace:
I20260619 21:16:09.972309 2421 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:09.981854 1735 rpcz_store.cc:276] 0619 21:16:08.668416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:08.670753 (+ 2337us) service_pool.cc:227] Handling call
0619 21:16:08.698684 (+ 27931us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:08.699376 (+ 692us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:08.699430 (+ 54us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:08.750511 (+ 51081us) log.cc:842] Serialized 26 byte log entry
0619 21:16:08.998027 (+247516us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:09.009613 (+ 11586us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:09.017511 (+ 7898us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:09.921992 (+904481us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b24001d92b8. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719b9eecf4 00007f719b9ecdfb 00007f719ba7b559 00007f719ba7b1e5 00007f719ba7a740 00007f719e76d578 00007f719e76dcd8 00007f719e7456e8 00007f719e740aea 00007f719e73ff36
0619 21:16:09.980657 (+ 58665us) raft_consensus.cc:1671] finished
0619 21:16:09.980706 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:09.981081 (+ 375us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":397,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":167,"spinlock_wait_cycles":100334976,"thread_start_us":222134,"threads_started":1,"wal-append.queue_time_us":284772}
I20260619 21:16:09.985097 1814 raft_consensus.cc:1261] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260619 21:16:09.973007 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.986569 2774 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:09.977041 1663 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:09.967010 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:09.995434 1740 raft_consensus.cc:2454] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:16:09.967531 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:09.998767 1814 pending_rounds.cc:76] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Aborting all ops after (but not including) 1
I20260619 21:16:10.000119 1814 pending_rounds.cc:97] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260619 21:16:10.001637 1814 raft_consensus.cc:2866] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:09.987272 2774 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.007203 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:09.933372 2703 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:10.007927 1814 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:09.979526 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.009210 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.010283 1814 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:10.011739 2688 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.012876 2757 consensus_queue.cc:1046] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:10.021272 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.021970 1814 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:10.022837 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.023398 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.023692 1814 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:10.025523 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:10.014178 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.026257 1663 raft_consensus.cc:2454] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 0.
I20260619 21:16:10.025077 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.029462 1814 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:10.017834 1783 leader_election.cc:302] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.010191 2703 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:10.030918 2774 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.018802 2711 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:10.032091 2711 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.032209 2684 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.032629 2711 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:10.030478 1635 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:10.037844 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:10.038130 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:10.038234 2775 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:16:10.035687 1710 leader_election.cc:341] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:10.018913 2559 consensus_queue.cc:1035] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260619 21:16:10.046413 2683 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.047192 2734 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260619 21:16:10.026335 2710 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:10.014732 2564 consensus_queue.cc:1035] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260619 21:16:10.006841 2693 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.056020 1635 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:10.057258 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.063217 1740 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:10.030946 2703 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:10.058787 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.041815 2684 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.037498 2673 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:10.066231 1710 consensus_peers.cc:487] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:16:10.035375 1811 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:10.048965 2774 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.051663 2736 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:10.071235 1635 leader_election.cc:341] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:10.075992 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.056600 2585 leader_election.cc:288] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.066032 2684 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.073980 2559 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:10.069562 2628 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.055958 2702 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.108217 2771 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.078270 2777 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260619 21:16:10.105516 2765 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.077823 2673 raft_consensus.cc:2781] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:16:10.112777 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.135710 1817 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
W20260619 21:16:10.115339 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 77834.6us
I20260619 21:16:10.137112 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:10.137212 1782 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:10.117125 1710 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.103302 2559 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.140581 1710 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.141731 1710 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.142700 1742 raft_consensus.cc:1226] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:10.142583 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.144469 1782 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:10.143362 1739 raft_consensus.cc:2454] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:16:10.103585 2684 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:10.082623 2736 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.134785 2655 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.154434 1741 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:10.079483 2774 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.138358 2673 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.061710 2710 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.150262 2623 leader_election.cc:288] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.162858 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:10.140034 2765 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.172044 1662 raft_consensus.cc:1261] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260619 21:16:10.172521 1662 pending_rounds.cc:76] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Aborting all ops after (but not including) 2
I20260619 21:16:10.172891 1662 pending_rounds.cc:97] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260619 21:16:10.172597 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.173252 1662 raft_consensus.cc:2866] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:10.149271 2652 raft_consensus.cc:2781] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:10.177448 2652 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.177980 2652 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:10.149375 2660 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:10.143627 2559 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:10.130424 2708 consensus_queue.cc:1035] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260619 21:16:10.040411 2775 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.134166 2771 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.161931 2606 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260619 21:16:10.219179 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 153032us
I20260619 21:16:10.216524 1635 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:10.219609 1859 raft_consensus.cc:348] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.225301 1859 raft_consensus.cc:374] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:10.225726 1859 raft_consensus.cc:729] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:10.220230 1783 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:10.226784 1859 consensus_queue.cc:260] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:10.244784 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 92267.7us
I20260619 21:16:10.167850 2564 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.163249 2673 raft_consensus.cc:3037] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:10.192674 2660 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.208535 2672 raft_consensus.cc:3032] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:10.169356 2777 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.228065 2421 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.167852 2774 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.166781 2623 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.195320 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:10.218081 2688 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.218291 2730 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.292307 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.356117 1814 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:10.293422 1710 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:10.357635 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 64337.6us
I20260619 21:16:10.189724 2765 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:10.360908 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 112503us
I20260619 21:16:10.163408 2655 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.180855 2600 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.290362 2564 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.291653 2757 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.371670 1859 ts_tablet_manager.cc:1428] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.395s user 0.006s sys 0.006s
W20260619 21:16:10.354897 2779 log.cc:924] Time spent T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.250s user 0.000s sys 0.002s
I20260619 21:16:10.397805 2784 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.333464 2660 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:10.362885 2623 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.404129 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.354480 2421 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.333595 2672 raft_consensus.cc:729] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:10.355674 2709 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.305670 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:10.363969 2693 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
PC: @ 0x0 (unknown)
I20260619 21:16:10.367503 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.376142 2757 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.376473 2564 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:10.452564 2757 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:10.425900 1664 raft_consensus.cc:2379] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:10.427114 2672 consensus_queue.cc:260] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.469523 1859 tablet_bootstrap.cc:492] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:10.429760 2775 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.402287 2702 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.470667 2702 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.471307 2702 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.430547 2655 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:10.434007 2780 log.cc:924] Time spent T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.184s user 0.000s sys 0.001s
I20260619 21:16:10.438896 2600 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.444492 1708 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:10.449054 2786 log.cc:924] Time spent T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
I20260619 21:16:10.415771 2623 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
W20260619 21:16:10.451695 2698 consensus_meta.cc:287] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.561s user 0.009s sys 0.004s
I20260619 21:16:10.363837 2777 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.418551 2421 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:10.368453 2730 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.368300 2688 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.460815 2683 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.425997 2424 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:10.476126 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.477831 1708 leader_election.cc:302] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:10.419400 2784 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.455114 1662 raft_consensus.cc:3037] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:10.497428 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 249630us
W20260619 21:16:10.499419 1634 leader_election.cc:334] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.990s (SENT)
I20260619 21:16:10.467343 1811 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:10.510211 1740 raft_consensus.cc:2454] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:10.481032 2655 consensus_queue.cc:1035] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260619 21:16:10.549110 1710 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.539470 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:10.550150 1668 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 6
I20260619 21:16:10.466578 2709 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.502992 2702 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:10.541934 2621 consensus_meta.cc:287] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.543s user 0.012s sys 0.001s
I20260619 21:16:10.523228 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:10.559394 1663 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:10.517122 1708 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:10.502427 2771 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.561908 2719 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:10.563107 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.511675 2784 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:10.539942 2697 raft_consensus.cc:2781] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:16:10.568393 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:10.572086 1740 raft_consensus.cc:3037] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:10.567487 1710 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.487782 2698 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.543937 2688 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.564101 1735 raft_consensus.cc:2379] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:10.576895 1742 raft_consensus.cc:1226] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
W20260619 21:16:10.564539 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 47535.1us
I20260619 21:16:10.589805 2719 raft_consensus.cc:2781] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:10.510550 2424 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.596342 2702 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.562654 2585 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.571784 2697 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.625465 2730 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.566771 2726 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:10.508527 2683 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:10.647658 2792 log.cc:924] Time spent T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.161s user 0.000s sys 0.001s
I20260619 21:16:10.647161 2585 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:10.650349 2703 consensus_meta.cc:287] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.580s user 0.012s sys 0.019s
I20260619 21:16:10.572968 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:10.629817 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.668524 1740 raft_consensus.cc:2454] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
W20260619 21:16:10.671908 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 74218us
I20260619 21:16:10.565510 2621 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.643375 1708 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:10.679693 2703 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.683874 2585 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.669842 2726 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.686298 2703 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.686681 2726 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:10.636924 2719 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:10.688467 1814 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
W20260619 21:16:10.688719 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 45442.2us
I20260619 21:16:10.689996 1708 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:10.690791 2621 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:16:10.692492 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.693097 2698 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.693357 1816 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:10.694658 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
*** SIGABRT (@0x3e80000018c) received by PID 396 (TID 0x7f71977c8880) from PID 396; stack trace: ***
@ 0x29bbc0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x29e2b0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
I20260619 21:16:10.696244 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.690810 1811 raft_consensus.cc:2454] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:16:10.653596 2697 raft_consensus.cc:3037] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:10.698549 1710 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:10.701169 2785 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.207s user 0.001s sys 0.000s
I20260619 21:16:10.701380 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:10.701843 1710 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.683126 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.712754 1782 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:16:10.717305 2652 consensus_meta.cc:287] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.539s user 0.006s sys 0.012s
I20260619 21:16:10.717880 2775 raft_consensus.cc:2781] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:10.723749 1783 leader_election.cc:302] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.699081 2708 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:10.698585 2719 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:10.653479 2702 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.727442 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.733798 2764 consensus_queue.cc:1046] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260619 21:16:10.734403 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.721525 2773 raft_consensus.cc:2781] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260619 21:16:10.735879 1710 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:10.706748 2788 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:10.721508 2775 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:10.732512 1668 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:10.709507 1816 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:10.740411 2797 raft_consensus.cc:2781] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:10.726451 2736 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.741186 2424 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.742070 2797 raft_consensus.cc:686] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:10.717677 2699 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:10.744015 1710 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:10.691921 2777 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.747051 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:10.722419 2698 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.749601 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.749105 1708 leader_election.cc:302] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:10.758457 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.728335 2621 raft_consensus.cc:2781] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:10.758517 1708 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:10.770641 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.772203 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.769809 2699 raft_consensus.cc:2781] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 21:16:10.745913 2708 consensus_queue.cc:1035] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260619 21:16:10.774111 2699 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.774680 2699 raft_consensus.cc:3037] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:10.762152 2788 raft_consensus.cc:686] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:10.778702 2702 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.779740 2710 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.722482 2652 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.735352 2789 consensus_queue.cc:1035] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260619 21:16:10.787726 2709 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.788440 2652 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.790395 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:10.793833 2800 log.cc:924] Time spent T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.115s user 0.000s sys 0.001s
I20260619 21:16:10.795029 2625 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:10.762382 2777 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:10.762218 2621 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.747311 2773 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260619 21:16:10.810492 2700 consensus_meta.cc:287] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.852s user 0.006s sys 0.018s
I20260619 21:16:10.811043 2700 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:10.816725 2711 consensus_meta.cc:287] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.773s user 0.000s sys 0.021s
I20260619 21:16:10.818859 1812 raft_consensus.cc:2354] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:10.759385 2765 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260619 21:16:10.758946 2771 consensus_queue.cc:1035] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260619 21:16:10.747623 2775 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:10.833374 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.801342 2777 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:10.847436 2673 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.803017 2708 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:10.781889 2698 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.803819 2621 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:10.809206 2773 raft_consensus.cc:3037] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:10.851284 2559 consensus_meta.cc:287] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.643s user 0.022s sys 0.010s
I20260619 21:16:10.745596 2797 consensus_queue.cc:237] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.866811 2708 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:10.867358 2708 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
@ 0x7f719eec3980 (unknown) at ??:0
I20260619 21:16:10.828135 2710 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:10.873960 2673 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.779934 2788 consensus_queue.cc:237] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.878106 2559 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.809861 2764 consensus_queue.cc:1035] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260619 21:16:10.905049 1860 tablet_bootstrap.cc:492] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:10.879460 2660 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.922518 1741 raft_consensus.cc:1226] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:10.922485 2623 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.926187 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.907260 1860 ts_tablet_manager.cc:1397] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.194s user 0.015s sys 0.025s
I20260619 21:16:10.857810 2777 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:10.928156 1816 raft_consensus.cc:1226] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260619 21:16:10.926123 2757 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.927124 1738 raft_consensus.cc:2362] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2. Re-sending same reply.
I20260619 21:16:10.937927 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.938757 1738 raft_consensus.cc:2454] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:16:10.939841 1739 raft_consensus.cc:1261] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:10.939940 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:10.926906 2700 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:10.942142 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.944722 2559 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.904795 2673 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:10.916411 1811 raft_consensus.cc:1226] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:10.916904 2708 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.952514 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:10.878578 2683 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:10.900667 2710 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.896842 2564 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.876552 2699 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.817328 2711 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.004339 2564 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.013794 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.014294 2527 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:11.014961 2527 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:10.964286 2673 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.015975 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.016064 1860 raft_consensus.cc:348] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.020980 1860 raft_consensus.cc:374] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:11.021246 1860 raft_consensus.cc:729] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:11.017154 1737 raft_consensus.cc:1226] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 5. Ops: []
I20260619 21:16:11.022081 1860 consensus_queue.cc:260] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.028034 2621 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.029834 1662 raft_consensus.cc:2454] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:11.030416 2698 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.059959 1860 ts_tablet_manager.cc:1428] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.132s user 0.008s sys 0.000s
I20260619 21:16:11.065351 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 517) took 1146 ms. Trace:
I20260619 21:16:11.065824 1815 rpcz_store.cc:276] 0619 21:16:09.918894 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:09.919066 (+ 172us) service_pool.cc:227] Handling call
0619 21:16:09.942000 (+ 22934us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:09.957726 (+ 15726us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.065063 (+1107337us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b54000f96d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:11.065264 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":34348288}
I20260619 21:16:11.067726 1738 raft_consensus.cc:2379] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:11.073838 2711 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.074518 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.075568 1738 raft_consensus.cc:2379] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:11.036132 2702 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:11.031874 2625 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.079984 2757 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.080320 2699 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.035534 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.083619 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.085132 1742 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 3
I20260619 21:16:11.084244 2702 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
@ 0x7f71984b8e87 gsignal at ??:0
I20260619 21:16:11.099985 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.101177 1738 raft_consensus.cc:2396] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260619 21:16:11.035125 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.084780 2625 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.068735 2660 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.122915 2683 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.123015 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.123864 2683 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.124598 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.125795 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.126344 1738 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:11.127460 2683 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.105738 1735 raft_consensus.cc:3037] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
W20260619 21:16:11.140486 1812 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 533) took 1173 ms (client timeout 1553 ms). Trace:
W20260619 21:16:11.141083 1812 rpcz_store.cc:269] 0619 21:16:09.966866 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:09.969702 (+ 2836us) service_pool.cc:227] Handling call
0619 21:16:11.140211 (+1170509us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b540023e6d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:11.140406 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1923566976}
I20260619 21:16:11.145630 1860 tablet_bootstrap.cc:492] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:11.146859 2777 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.149773 1736 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:11.153817 2422 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:11.029886 2784 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.169173 1738 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:11.155269 2621 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.180639 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.181488 1738 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:11.154072 2710 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.112150 1737 raft_consensus.cc:2362] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4. Re-sending same reply.
I20260619 21:16:11.189905 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.194813 1783 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:11.196368 1783 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.204380 1738 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
W20260619 21:16:11.188108 2726 consensus_meta.cc:287] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.501s user 0.010s sys 0.005s
I20260619 21:16:11.180991 2702 leader_election.cc:288] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.163630 2773 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.132687 2719 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:11.213486 2767 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s user 0.012s sys 0.030s
I20260619 21:16:11.203148 2620 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:11.214468 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.217227 2606 raft_consensus.cc:2781] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:11.211676 1742 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:11.245114 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.246031 1743 raft_consensus.cc:2379] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:16:11.223851 2697 consensus_meta.cc:287] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.512s user 0.006s sys 0.009s
I20260619 21:16:11.224185 2777 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.215565 1783 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.235492 1859 tablet_bootstrap.cc:654] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: Neither blocks nor log segments found. Creating new log.
W20260619 21:16:11.242399 2684 consensus_meta.cc:287] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.083s user 0.016s sys 0.007s
I20260619 21:16:11.245766 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:11.220921 1635 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.622s user 0.024s sys 0.017s
I20260619 21:16:11.246548 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 479) took 1104 ms. Trace:
I20260619 21:16:11.220816 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:16:11.226744 2767 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s user 0.012s sys 0.030s
I20260619 21:16:11.251386 1783 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:11.253791 1635 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.655s user 0.024s sys 0.017s
I20260619 21:16:11.253919 1743 rpcz_store.cc:276] 0619 21:16:10.141691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.141832 (+ 141us) service_pool.cc:227] Handling call
0619 21:16:11.244927 (+1103095us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:11.246495 (+ 1568us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5435392}
W20260619 21:16:11.393029 1633 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.669s user 0.013s sys 0.032s
W20260619 21:16:11.393501 1633 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.766s user 0.013s sys 0.033s
I20260619 21:16:11.394192 1743 raft_consensus.cc:1226] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:11.394851 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 490) took 1093 ms. Trace:
I20260619 21:16:11.395203 1743 rpcz_store.cc:276] 0619 21:16:10.301019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.393620 (+1092601us) service_pool.cc:227] Handling call
0619 21:16:11.394011 (+ 391us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.394551 (+ 540us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.394776 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.396486 1743 raft_consensus.cc:1226] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 3. Ops: []
I20260619 21:16:11.397015 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 494) took 1095 ms. Trace:
I20260619 21:16:11.397291 1743 rpcz_store.cc:276] 0619 21:16:10.301915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.396046 (+1094131us) service_pool.cc:227] Handling call
0619 21:16:11.396321 (+ 275us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.396788 (+ 467us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.396970 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.398572 1743 raft_consensus.cc:1261] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:11.400519 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:11.238580 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 494586us
I20260619 21:16:11.401528 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.402583 1814 raft_consensus.cc:2379] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:11.403649 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.404359 1814 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:11.255376 2602 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260619 21:16:11.259634 2421 consensus_meta.cc:287] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.759s user 0.007s sys 0.014s
I20260619 21:16:11.263327 2805 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:11.259594 2688 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.590s user 0.006s sys 0.012s
I20260619 21:16:11.259909 1737 raft_consensus.cc:3037] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:11.274113 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 477) took 1133 ms. Trace:
I20260619 21:16:11.650357 1736 rpcz_store.cc:276] 0619 21:16:10.140859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.144481 (+ 3622us) service_pool.cc:227] Handling call
0619 21:16:11.273808 (+1129327us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54002eefd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:11.274008 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1953062784}
I20260619 21:16:11.250079 2697 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.266973 2773 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.256287 2684 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.285763 1860 tablet_bootstrap.cc:654] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:11.292433 2620 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:11.284243 2784 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:11.299079 1632 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.648s user 0.008s sys 0.038s
W20260619 21:16:11.656200 1632 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.056s user 0.009s sys 0.038s
I20260619 21:16:11.657995 2693 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:11.658124 2684 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.658198 2697 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.658661 2684 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.658547 2693 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.658850 2620 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.659134 2684 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.660696 2693 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.660831 2620 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.294340 1735 raft_consensus.cc:2454] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:11.297667 2719 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.315798 1744 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:11.318387 2672 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:11.322504 2803 log.cc:924] Time spent T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260619 21:16:11.323055 2625 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:11.332204 1708 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.533s user 0.005s sys 0.034s
I20260619 21:16:11.346580 1811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 546) took 1081 ms. Trace:
I20260619 21:16:11.347666 1741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 484) took 1189 ms. Trace:
I20260619 21:16:11.348441 2673 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.333276 1664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 460) took 1406 ms. Trace:
W20260619 21:16:11.358217 2802 log.cc:924] Time spent T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.091s user 0.000s sys 0.003s
I20260619 21:16:11.661664 1735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 512) took 1150 ms. Trace:
I20260619 21:16:11.359084 2623 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.362241 2527 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:11.363794 2775 consensus_meta.cc:287] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.505s user 0.013s sys 0.002s
W20260619 21:16:11.393018 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 141698us
I20260619 21:16:11.215716 2726 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.681217 1818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 542) took 1420 ms. Trace:
I20260619 21:16:11.681378 1783 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.250625 2606 raft_consensus.cc:686] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:11.682519 2620 raft_consensus.cc:2781] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:11.683063 2620 raft_consensus.cc:686] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:11.683218 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 544) took 1420 ms. Trace:
I20260619 21:16:11.683333 2606 consensus_queue.cc:237] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.683919 2620 consensus_queue.cc:237] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.413394 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.683558 1817 rpcz_store.cc:276] 0619 21:16:10.262795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.441431 (+178636us) service_pool.cc:227] Handling call
0619 21:16:10.473557 (+ 32126us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:10.476642 (+ 3085us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:10.476672 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:10.481793 (+ 5121us) log.cc:842] Serialized 26 byte log entry
0619 21:16:10.959270 (+477477us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:10.959308 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:10.959383 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:11.357461 (+398078us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719e740bee 00007f719e73ff36 00007f719e73fb58 00007f719e749551 00007f719e749509 00007f719e749499
0619 21:16:11.682920 (+325459us) raft_consensus.cc:1671] finished
0619 21:16:11.682939 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:11.683156 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2684,"raft.queue_time_us":5175,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":6831,"spinlock_wait_cycles":2917504,"thread_start_us":448129,"threads_started":1,"wal-append.queue_time_us":759840}
I20260619 21:16:11.685636 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.671518 1811 rpcz_store.cc:276] 0619 21:16:10.264692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.744869 (+480177us) service_pool.cc:227] Handling call
0619 21:16:10.799832 (+ 54963us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:10.964111 (+164279us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.346182 (+382071us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b540031f6d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:11.346483 (+ 301us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":229049216}
I20260619 21:16:11.422111 1809 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:11.437156 2422 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.459154 2602 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:11.423094 1782 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:16:11.622838 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 185689us
W20260619 21:16:11.739738 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 316686us
I20260619 21:16:11.464958 2421 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.614322 2460 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.627544 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.628515 2759 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:11.628579 2810 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:11.749473 2810 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.750516 2460 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.751967 2810 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.629513 2683 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.752862 2683 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.753505 2797 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:11.753916 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.753998 2797 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.754611 1809 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:11.632017 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 488) took 1331 ms. Trace:
I20260619 21:16:11.714540 2623 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:11.654253 1736 raft_consensus.cc:1226] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:11.655155 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.762072 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 496) took 1459 ms. Trace:
I20260619 21:16:11.762537 1736 rpcz_store.cc:276] 0619 21:16:10.302356 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.651280 (+1348924us) service_pool.cc:227] Handling call
0619 21:16:11.651592 (+ 312us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.761707 (+110115us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.761995 (+ 288us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.655202 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:11.646183 2688 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.980s user 0.006s sys 0.012s
I20260619 21:16:11.663074 1667 raft_consensus.cc:1261] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:11.767588 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.768399 1667 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:11.666810 1737 raft_consensus.cc:2454] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:11.635591 2771 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.653544 2805 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.669237 1563 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37734 (request call id 81) took 9635 ms. Trace:
I20260619 21:16:11.755203 2673 raft_consensus.cc:2781] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader election won for term 6
W20260619 21:16:11.671474 1708 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.872s user 0.005s sys 0.034s
I20260619 21:16:11.671618 1741 rpcz_store.cc:276] 0619 21:16:10.158177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.158307 (+ 130us) service_pool.cc:227] Handling call
0619 21:16:10.181311 (+ 23004us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:10.923170 (+741859us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.347370 (+424200us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54002ee0d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:11.347589 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1703961344}
I20260619 21:16:11.646570 2795 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:11.679280 1735 rpcz_store.cc:276] 0619 21:16:10.511156 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.031354 (+520198us) service_pool.cc:227] Handling call
0619 21:16:11.661597 (+630243us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.681622 1818 rpcz_store.cc:276] 0619 21:16:10.260831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.261010 (+ 179us) service_pool.cc:227] Handling call
0619 21:16:10.332096 (+ 71086us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:10.493718 (+161622us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:10.493763 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:10.507834 (+ 14071us) log.cc:842] Serialized 26 byte log entry
0619 21:16:10.964422 (+456588us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:10.964471 (+ 49us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:10.964559 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:11.681001 (+716442us) raft_consensus.cc:1671] finished
0619 21:16:11.681021 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:11.681155 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8385,"raft.queue_time_us":110217,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":22593,"thread_start_us":587651,"threads_started":2,"wal-append.queue_time_us":721557}
I20260619 21:16:11.652969 2806 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.677714 1664 rpcz_store.cc:276] 0619 21:16:09.925753 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:09.927940 (+ 2187us) service_pool.cc:227] Handling call
0619 21:16:11.328886 (+1400946us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54000aa354. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:11.332340 (+ 3454us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1036180224}
I20260619 21:16:11.662050 2774 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:11.687089 1740 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:11.405102 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 495) took 1103 ms. Trace:
I20260619 21:16:11.686913 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.680030 2775 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.476781 2713 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:11.685738 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.682542 2726 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.664288 2684 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.691859 1812 raft_consensus.cc:1261] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260619 21:16:11.703964 1744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 474) took 1564 ms (client timeout 1551 ms). Trace:
W20260619 21:16:11.815210 1744 rpcz_store.cc:269] 0619 21:16:10.139232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.139984 (+ 752us) service_pool.cc:227] Handling call
0619 21:16:11.703720 (+1563736us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540022c754. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:11.703875 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1967453952}
I20260619 21:16:11.712512 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.816612 1814 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:11.727313 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 489) took 1426 ms. Trace:
I20260619 21:16:11.818010 1742 rpcz_store.cc:276] 0619 21:16:10.300764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.255828 (+955064us) service_pool.cc:227] Handling call
0619 21:16:11.256163 (+ 335us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:11.256524 (+ 361us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:11.256562 (+ 38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:11.257101 (+ 539us) log.cc:842] Serialized 26 byte log entry
0619 21:16:11.583411 (+326310us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:11.723404 (+139993us) log.cc:842] Serialized 14 byte log entry
0619 21:16:11.723697 (+ 293us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.727039 (+ 3342us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:11.727087 (+ 48us) raft_consensus.cc:1671] finished
0619 21:16:11.727099 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:11.727236 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2031,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":12393,"thread_start_us":411636,"threads_started":2,"wal-append.queue_time_us":367274}
I20260619 21:16:11.747720 1710 leader_election.cc:398] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:11.820374 1710 leader_election.cc:401] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260619 21:16:11.749380 2759 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.755314 1738 rpcz_store.cc:276] 0619 21:16:10.300439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.242441 (+942002us) service_pool.cc:227] Handling call
0619 21:16:11.242815 (+ 374us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:11.243190 (+ 375us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:11.243225 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:11.245207 (+ 1982us) log.cc:842] Serialized 26 byte log entry
0619 21:16:11.598318 (+353111us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:16:11.600621 (+ 2303us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.604999 (+ 4378us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:11.631646 (+ 26647us) raft_consensus.cc:1671] finished
0619 21:16:11.631678 (+ 32us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:11.631913 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":555,"raft.queue_time_us":628,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4818,"thread_start_us":342902,"threads_started":1,"wal-append.queue_time_us":379279}
I20260619 21:16:11.648056 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.648289 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.828184 1669 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:11.754978 2683 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.768782 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.752743 2460 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.743640 2602 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:11.772063 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 540) took 1514 ms. Trace:
I20260619 21:16:11.738943 2756 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:11.654597 2809 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:11.783607 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 1.01283e+06us
I20260619 21:16:11.791705 1668 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:11.839938 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.796144 1743 rpcz_store.cc:276] 0619 21:16:10.302131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.397923 (+1095792us) service_pool.cc:227] Handling call
0619 21:16:11.398197 (+ 274us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.404866 (+ 6669us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.405054 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.840727 1668 raft_consensus.cc:2379] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:11.771530 2623 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.775254 2810 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:11.805230 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.747354 2422 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.808822 2774 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.852438 2759 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.853200 2422 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.835383 2726 raft_consensus.cc:2726] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:11.810494 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.854339 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.854980 1664 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:11.824404 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.856235 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.856781 1818 raft_consensus.cc:3037] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:11.859724 1740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 507) took 1489 ms (client timeout 1810 ms). Trace:
W20260619 21:16:11.860126 1740 rpcz_store.cc:269] 0619 21:16:10.369945 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.795485 (+425540us) service_pool.cc:227] Handling call
0619 21:16:11.859535 (+1064050us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b540032cb54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:11.859665 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1960132864}
I20260619 21:16:11.865746 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.866247 1818 raft_consensus.cc:2454] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:11.866775 1817 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:11.867144 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.869170 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.870012 1662 raft_consensus.cc:3037] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:11.870414 2623 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:11.831131 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 90705.9us
I20260619 21:16:11.872826 1710 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.873934 2422 consensus_queue.cc:1046] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:11.874536 2797 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:11.874933 1859 tablet_bootstrap.cc:492] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: No bootstrap required, opened a new log
I20260619 21:16:11.875562 1859 ts_tablet_manager.cc:1397] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: Time spent bootstrapping tablet: real 1.413s user 0.019s sys 0.047s
I20260619 21:16:11.836349 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.780375 1563 rpcz_store.cc:276] 0619 21:16:02.033633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:02.033845 (+ 212us) service_pool.cc:227] Handling call
0619 21:16:11.669135 (+9635290us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0619 21:16:07.425478 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0619 21:16:07.571777 (+146299us) write_op.cc:432] Acquiring schema lock in shared mode
0619 21:16:07.578725 (+ 6948us) write_op.cc:435] Acquired schema lock
0619 21:16:07.578754 (+ 29us) tablet.cc:660] Decoding operations
0619 21:16:07.579580 (+ 826us) write_op.cc:620] Acquiring the partition lock for write op
0619 21:16:07.579677 (+ 97us) write_op.cc:641] Partition lock acquired for write op
0619 21:16:07.579706 (+ 29us) tablet.cc:683] Acquiring locks for 5 operations
0619 21:16:07.580312 (+ 606us) tablet.cc:699] Row locks acquired
0619 21:16:07.580358 (+ 46us) write_op.cc:260] PREPARE: finished
0619 21:16:07.580521 (+ 163us) write_op.cc:270] Start()
0619 21:16:07.580669 (+ 148us) write_op.cc:276] Timestamp: P: 1781903767580463 usec, L: 0
0619 21:16:07.580691 (+ 22us) op_driver.cc:331] REPLICATION: starting
0619 21:16:07.581381 (+ 690us) log.cc:842] Serialized 2048 byte log entry
0619 21:16:07.935503 (+354122us) op_driver.cc:444] REPLICATION: finished
0619 21:16:08.084877 (+149374us) write_op.cc:301] APPLY: starting
0619 21:16:08.085027 (+ 150us) tablet.cc:1365] starting BulkCheckPresence
0619 21:16:08.085217 (+ 190us) tablet.cc:1368] finished BulkCheckPresence
0619 21:16:08.085339 (+ 122us) tablet.cc:1370] starting ApplyRowOperation cycle
0619 21:16:08.112483 (+ 27144us) tablet.cc:1381] finished ApplyRowOperation cycle
0619 21:16:08.112681 (+ 198us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0619 21:16:08.112737 (+ 56us) write_op.cc:312] APPLY: finished
0619 21:16:08.113042 (+ 305us) log.cc:842] Serialized 46 byte log entry
0619 21:16:08.113217 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:08.113334 (+ 117us) write_op.cc:454] Released schema lock
0619 21:16:08.113376 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0619 21:16:08.113635 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0619 21:16:08.113664 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49387,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":141444,"apply.run_cpu_time_us":3227,"apply.run_wall_time_us":36778,"num_ops":5,"prepare.queue_time_us":207379,"prepare.run_cpu_time_us":43322,"prepare.run_wall_time_us":347990,"raft.queue_time_us":165308,"raft.run_cpu_time_us":29158,"raft.run_wall_time_us":100000,"replication_time_us":354698,"thread_start_us":447044,"threads_started":4,"wal-append.queue_time_us":172610,"wal-append.run_cpu_time_us":12818,"wal-append.run_wall_time_us":1372568}]]}
I20260619 21:16:11.840699 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.842270 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.834928 1739 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:11.797271 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.881470 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.882565 1663 raft_consensus.cc:2379] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:11.883458 1736 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:11.874894 2726 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:11.791419 2673 raft_consensus.cc:686] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [term 6 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:11.802654 1667 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:11.791096 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.798112 1783 leader_election.cc:302] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.844515 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:11.918716 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 120696us
I20260619 21:16:11.782564 2771 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.850497 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.833168 2806 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
@ 0x7f71984ba7f1 abort at ??:0
@ 0x2a0007 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
I20260619 21:16:11.851051 2810 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.834399 2713 raft_consensus.cc:686] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:11.779129 1665 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:11.961683 1859 raft_consensus.cc:348] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.962380 1859 raft_consensus.cc:374] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:11.962630 1859 raft_consensus.cc:729] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Initialized, Role: FOLLOWER
I20260619 21:16:11.963860 1710 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.822739 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:11.965109 1815 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:11.843557 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:11.966236 1669 raft_consensus.cc:3037] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:11.967319 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 514) took 1429 ms. Trace:
I20260619 21:16:11.967789 1742 rpcz_store.cc:276] 0619 21:16:10.537987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.919731 (+1381744us) service_pool.cc:227] Handling call
0619 21:16:11.966647 (+ 46916us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.966930 (+ 283us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:11.966966 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:11.967017 (+ 51us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:11.967061 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.967128 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:11.967250 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.875164 2756 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.894153 1816 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:11.909845 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.912621 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.970870 1744 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:11.970588 1783 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:11.914031 1739 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 517) took 1336 ms (client timeout 1716 ms). Trace:
W20260619 21:16:11.972045 1739 rpcz_store.cc:269] 0619 21:16:10.670246 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.940802 (+270556us) service_pool.cc:227] Handling call
0619 21:16:11.913789 (+972987us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b5400092c54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:11.913949 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2051206016}
I20260619 21:16:11.973405 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.974929 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.975857 1739 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:11.915512 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.922178 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.905349 1740 raft_consensus.cc:1261] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:11.877827 2809 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.925601 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.927765 2774 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.929980 2775 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:11.791877 2795 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.838762 1813 rpcz_store.cc:276] 0619 21:16:10.257703 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:10.257868 (+ 165us) service_pool.cc:227] Handling call
0619 21:16:11.297377 (+1039509us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:11.297625 (+ 248us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.297858 (+ 233us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:11.607388 (+309530us) log.cc:842] Serialized 14 byte log entry
0619 21:16:11.771592 (+164204us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:11.771668 (+ 76us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:11.771726 (+ 58us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.771813 (+ 87us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:11.771971 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25814,"prepare.queue_time_us":28664,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":823,"raft.queue_time_us":26290,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4400,"spinlock_wait_cycles":16619264,"thread_start_us":422351,"threads_started":2}
I20260619 21:16:11.996330 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:11.997028 1742 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:11.999078 1739 raft_consensus.cc:1226] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:11.999802 1739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 515) took 1444 ms. Trace:
I20260619 21:16:12.000074 1739 rpcz_store.cc:276] 0619 21:16:10.554974 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.979238 (+1424264us) service_pool.cc:227] Handling call
0619 21:16:11.979597 (+ 359us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.999578 (+ 19981us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.999754 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:11.935429 1814 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:12.001027 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:11.941538 2797 raft_consensus.cc:2781] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:11.889094 2422 raft_consensus.cc:3032] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:11.954540 2421 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:11.957664 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 92833.8us
I20260619 21:16:11.977977 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 506) took 1608 ms. Trace:
I20260619 21:16:11.963207 1859 consensus_queue.cc:260] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:11.839138 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.013036 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.013681 1737 raft_consensus.cc:2362] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2. Re-sending same reply.
I20260619 21:16:12.014523 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.015511 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.016036 1737 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:11.983003 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.017256 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 53480.5us
I20260619 21:16:12.017683 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.017911 1710 leader_election.cc:341] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:12.018492 1667 raft_consensus.cc:2379] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:12.018548 1710 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.959937 1661 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.019510 1710 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.020254 2716 raft_consensus.cc:2781] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:12.020391 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.020550 1710 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:11.984001 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.970307 1743 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:11.918038 1812 raft_consensus.cc:3037] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:11.982565 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:11.921306 2726 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:12.003150 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:11.909227 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.007428 1740 rpcz_store.cc:276] 0619 21:16:10.369619 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.860891 (+1491272us) service_pool.cc:227] Handling call
0619 21:16:11.888286 (+ 27395us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:11.888536 (+ 250us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.977688 (+ 89152us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:11.977910 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":60270976}
I20260619 21:16:12.010329 1859 ts_tablet_manager.cc:1428] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: Time spent starting tablet: real 0.134s user 0.005s sys 0.001s
I20260619 21:16:12.049418 1668 raft_consensus.cc:2454] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:12.034801 1783 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.013207 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 516) took 1442 ms. Trace:
I20260619 21:16:12.001783 1739 raft_consensus.cc:2379] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:12.041527 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.041766 1663 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:12.050808 1742 rpcz_store.cc:276] 0619 21:16:10.570631 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.997670 (+1427039us) service_pool.cc:227] Handling call
0619 21:16:11.997977 (+ 307us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.003922 (+ 5945us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:12.003955 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.004009 (+ 54us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.004051 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.012984 (+ 8933us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b540001efd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:12.013030 (+ 46us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.013152 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13137664}
I20260619 21:16:12.040473 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.045787 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.055526 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.057335 1816 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:12.058776 1735 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.061951 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.062882 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.056243 1743 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:11.995368 2623 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:12.019519 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.072438 1742 raft_consensus.cc:2354] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 5. Current term is 6.
I20260619 21:16:12.048166 1818 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:12.072475 1667 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.074749 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.075528 1667 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:12.063696 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.028488 1635 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:12.076757 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.077472 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 49077.5us
I20260619 21:16:12.065351 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.077688 1667 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:12.066155 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.079640 1663 raft_consensus.cc:2379] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.071476 1744 raft_consensus.cc:2454] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:12.058835 2713 consensus_queue.cc:237] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.080440 1783 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.077891 1816 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:12.082445 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.082621 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.083626 1667 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.084333 2623 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:12.023085 2683 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260619 21:16:12.084782 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.085528 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.085974 1669 raft_consensus.cc:2454] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.088596 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.089618 1739 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
W20260619 21:16:12.070601 1710 leader_election.cc:341] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:12.084956 2772 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:12.093317 2772 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:12.093634 2772 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:12.093900 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 519) took 1391 ms. Trace:
I20260619 21:16:12.016798 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.094507 2809 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.095923 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.096728 1813 raft_consensus.cc:2354] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:12.084826 2623 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.097333 2810 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.087371 1635 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:12.099081 2795 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.099876 1635 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:12.100479 2771 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.100164 2602 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.100654 2683 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:12.065524 2726 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.101106 2683 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:12.101234 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.101541 2683 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:12.101876 1743 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:12.102145 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.102756 1668 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:12.103235 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 524) took 1301 ms. Trace:
I20260619 21:16:12.103760 1668 raft_consensus.cc:1226] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:12.105135 2810 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:12.021406 2673 consensus_queue.cc:237] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.104471 2734 consensus_queue.cc:1035] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:16:12.105545 2810 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:12.105935 2810 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:12.106350 2806 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.103576 1743 rpcz_store.cc:276] 0619 21:16:10.802269 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.102493 (+1300224us) service_pool.cc:227] Handling call
0619 21:16:12.102777 (+ 284us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.102933 (+ 156us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:12.102968 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.103009 (+ 41us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:12.103039 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.103087 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.103181 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:12.021095 1661 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.108428 1662 raft_consensus.cc:2454] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:12.109136 1742 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:12.110529 1662 raft_consensus.cc:1226] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:12.110872 1661 raft_consensus.cc:1226] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:12.020757 2716 raft_consensus.cc:686] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:12.114058 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.094269 1740 rpcz_store.cc:276] 0619 21:16:10.702827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.085695 (+1382868us) service_pool.cc:227] Handling call
0619 21:16:12.093398 (+ 7703us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.093584 (+ 186us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:12.093610 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.093656 (+ 46us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.093690 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.093743 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.093842 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:12.108472 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 525) took 1302 ms. Trace:
I20260619 21:16:12.117067 1743 rpcz_store.cc:276] 0619 21:16:10.805720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.107736 (+1302016us) service_pool.cc:227] Handling call
0619 21:16:12.108020 (+ 284us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.108189 (+ 169us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:12.108218 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.108254 (+ 36us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:12.108285 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.108332 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.108422 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:12.118516 1663 raft_consensus.cc:1261] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:12.119243 1783 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.021117 2719 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:12.120751 1816 raft_consensus.cc:1261] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:12.121454 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.007973 2422 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:12.130235 1739 raft_consensus.cc:2379] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 1: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:12.116844 1742 raft_consensus.cc:1261] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:12.023044 2756 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.064236 1665 raft_consensus.cc:3037] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.072774 2808 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:12.132730 2422 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.066401 2774 raft_consensus.cc:2781] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:12.132860 2808 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.152488 2422 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:12.153189 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.159253 1664 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:12.112423 2421 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260619 21:16:12.161935 1634 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.515s user 0.086s sys 0.107s
I20260619 21:16:12.240526 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:12.167433 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 48298.3us
I20260619 21:16:12.168133 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.171257 1818 raft_consensus.cc:2454] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:12.172350 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.175724 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:12.185461 1810 raft_consensus.cc:1226] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:12.205425 1710 leader_election.cc:302] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.072625 2697 raft_consensus.cc:2781] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:12.055137 2797 raft_consensus.cc:686] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:16:12.240716 1634 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s user 0.087s sys 0.108s
W20260619 21:16:12.354916 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 727623us
W20260619 21:16:12.096027 2824 log.cc:924] Time spent T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.089s user 0.000s sys 0.000s
I20260619 21:16:12.242859 1669 raft_consensus.cc:2454] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:12.243533 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.244581 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.129268 2719 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:12.264109 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.264256 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.269562 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.211242 2672 raft_consensus.cc:2781] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:12.266597 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.273627 1635 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:12.279541 2808 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.281056 2602 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.240583 2421 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:12.260526 1811 raft_consensus.cc:1226] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:12.240732 2774 raft_consensus.cc:686] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:12.285118 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.285462 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.299647 2716 consensus_queue.cc:237] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.295042 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.268445 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.306740 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 257187us
I20260619 21:16:12.383165 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.270934 2805 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.385665 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.367261 1809 consensus_meta.cc:287] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.608s user 0.011s sys 0.007s
I20260619 21:16:12.319490 1860 tablet_bootstrap.cc:492] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:12.297726 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.282020 2810 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.386771 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.404759 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:16:12.352751 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 147408us
I20260619 21:16:12.353891 1667 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.405932 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.406378 1710 leader_election.cc:398] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:12.406746 1710 leader_election.cc:401] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260619 21:16:12.406992 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.407714 1667 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.407298 2623 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.359448 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 518) took 1679 ms. Trace:
I20260619 21:16:12.369014 2759 raft_consensus.cc:2781] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:12.357883 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.351425 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.360858 2719 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.355300 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 511) took 1848 ms. Trace:
W20260619 21:16:12.361343 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 87791us
I20260619 21:16:12.411718 1663 raft_consensus.cc:3037] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:12.408870 1744 rpcz_store.cc:276] 0619 21:16:10.673720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.080357 (+1406637us) service_pool.cc:227] Handling call
0619 21:16:12.080701 (+ 344us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.080919 (+ 218us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:12.129719 (+ 48800us) log.cc:842] Serialized 14 byte log entry
0619 21:16:12.352664 (+222945us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.352709 (+ 45us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.352746 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.352808 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.352922 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1744,"prepare.queue_time_us":51142,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":2407,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":220817,"threads_started":1}
I20260619 21:16:12.362409 1740 raft_consensus.cc:2454] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 0.
I20260619 21:16:12.413311 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.359531 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.366325 2775 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:12.369102 2809 raft_consensus.cc:2781] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:12.415119 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.367551 1742 raft_consensus.cc:2379] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:12.416623 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:16:12.372519 1634 leader_election.cc:334] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 3.170s (SENT)
I20260619 21:16:12.373950 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.417806 1818 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:12.418663 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.255707 2606 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:12.317643 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 513) took 1785 ms. Trace:
I20260619 21:16:12.404881 2810 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.409582 1669 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.426990 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 543) took 1027 ms. Trace:
I20260619 21:16:12.427045 1782 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.427408 1810 rpcz_store.cc:276] 0619 21:16:11.399150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.414036 (+ 14886us) service_pool.cc:227] Handling call
0619 21:16:11.414413 (+ 377us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.350765 (+936352us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.426777 (+ 76012us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b540002fdd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:12.426935 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1772362112}
I20260619 21:16:12.379524 2672 raft_consensus.cc:686] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:12.369233 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.419621 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.431329 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.431802 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.442288 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.409209 2625 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260619 21:16:12.453109 1809 raft_consensus.cc:2396] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:12.453642 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 561) took 1033 ms. Trace:
I20260619 21:16:12.453979 1809 rpcz_store.cc:276] 0619 21:16:11.420575 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.753578 (+333003us) service_pool.cc:227] Handling call
0619 21:16:12.453564 (+699986us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:12.430162 2606 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.434867 1669 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:12.456341 2606 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.424139 1664 raft_consensus.cc:2454] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 0.
I20260619 21:16:12.458485 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.409622 2754 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:12.461619 2754 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.460906 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.463716 2754 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.409857 2673 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:12.464370 2673 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.465067 1744 raft_consensus.cc:1226] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 5. Current term is 6. Ops: []
I20260619 21:16:12.466379 2673 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.468503 1818 raft_consensus.cc:1226] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:12.469789 1740 raft_consensus.cc:2379] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:12.368314 2697 raft_consensus.cc:686] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:12.421291 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.422143 1738 rpcz_store.cc:276] 0619 21:16:10.532434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.885584 (+1353150us) service_pool.cc:227] Handling call
0619 21:16:11.885997 (+ 413us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.886247 (+ 250us) raft_consensus.cc:1520] Early marking committed up to index 3
0619 21:16:11.964651 (+ 78404us) log.cc:842] Serialized 14 byte log entry
0619 21:16:12.317317 (+352666us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.317369 (+ 52us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:12.317404 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.317473 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.317584 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3998,"prepare.queue_time_us":74220,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1872,"raft.queue_time_us":876,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3189,"thread_start_us":351394,"threads_started":1,"wal-append.queue_time_us":456302}
I20260619 21:16:12.380977 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.423445 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:12.423640 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 58411.2us
I20260619 21:16:12.376924 2421 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:12.420100 1736 rpcz_store.cc:276] 0619 21:16:10.507249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.884342 (+1377093us) service_pool.cc:227] Handling call
0619 21:16:11.965499 (+ 81157us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.965761 (+ 262us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:12.303529 (+337768us) log.cc:842] Serialized 14 byte log entry
0619 21:16:12.354318 (+ 50789us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.354405 (+ 87us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.354509 (+ 104us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.354878 (+ 369us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.355164 (+ 286us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30426,"prepare.queue_time_us":166100,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"raft.queue_time_us":112,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6719,"thread_start_us":324316,"threads_started":2,"wal-append.queue_time_us":109132}
W20260619 21:16:12.355589 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 100374us
I20260619 21:16:12.429878 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.480971 1668 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:12.428776 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.448165 1813 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:12.443531 1661 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.478657 1816 raft_consensus.cc:1226] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:12.416929 1860 ts_tablet_manager.cc:1397] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.272s user 0.040s sys 0.049s
I20260619 21:16:12.486232 1817 raft_consensus.cc:2379] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:16:12.487103 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 58805.2us
I20260619 21:16:12.430356 2809 raft_consensus.cc:686] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:12.503247 1810 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:12.508076 1634 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.477876 2623 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.507550 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.530746 1668 raft_consensus.cc:2379] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.486783 2754 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:12.477018 2602 consensus_queue.cc:1035] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260619 21:16:12.467906 2797 consensus_queue.cc:237] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.528070 2806 consensus_queue.cc:1046] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260619 21:16:12.497327 2774 consensus_queue.cc:237] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
@ 0x7f719b22adf4 google::LogMessage::SendToLog() at ??:0
I20260619 21:16:12.612160 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.584700 2672 consensus_queue.cc:237] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.532198 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.628785 1665 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.735807 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.586995 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 100903us
I20260619 21:16:12.737229 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.597901 2809 consensus_queue.cc:237] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:12.597762 2834 log.cc:924] Time spent T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.177s user 0.001s sys 0.000s
I20260619 21:16:12.665432 1811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 546) took 1265 ms. Trace:
I20260619 21:16:12.741158 1664 raft_consensus.cc:3037] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:12.611641 2850 log.cc:924] Time spent T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.126s user 0.000s sys 0.001s
I20260619 21:16:12.741621 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.640596 2719 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.742403 1739 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.743701 2719 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:12.662452 2855 log.cc:924] Time spent T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
W20260619 21:16:12.597340 2837 log.cc:924] Time spent T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.112s user 0.000s sys 0.001s
I20260619 21:16:12.666821 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 481) took 1017 ms. Trace:
I20260619 21:16:12.746212 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.746905 1736 raft_consensus.cc:1261] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:12.747345 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.747969 1667 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:12.748347 2734 raft_consensus.cc:3032] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:12.748708 2734 raft_consensus.cc:729] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:12.533192 2693 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:12.749063 2847 consensus_queue.cc:1035] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260619 21:16:12.592176 2830 log.cc:924] Time spent T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.278s user 0.000s sys 0.001s
I20260619 21:16:12.748806 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.749517 2734 consensus_queue.cc:260] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.592866 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:16:12.537230 2829 log.cc:924] Time spent T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.233s user 0.001s sys 0.001s
I20260619 21:16:12.750835 2734 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:12.751997 1660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 483) took 1089 ms. Trace:
I20260619 21:16:12.752377 1660 rpcz_store.cc:276] 0619 21:16:11.662564 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.664007 (+ 1443us) service_pool.cc:227] Handling call
0619 21:16:11.680061 (+ 16054us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:11.680478 (+ 417us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:11.680513 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:11.681056 (+ 543us) log.cc:842] Serialized 26 byte log entry
0619 21:16:12.112260 (+431204us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.121344 (+ 9084us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.133829 (+ 12485us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:12.751789 (+617960us) raft_consensus.cc:1671] finished
0619 21:16:12.751810 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.751938 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":8798,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2310,"thread_start_us":375946,"threads_started":1,"wal-append.queue_time_us":589238}
I20260619 21:16:12.754302 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.750773 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.755019 1660 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:12.755760 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.755846 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.756379 1667 raft_consensus.cc:3037] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:12.758183 1739 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.558314 2856 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:12.759196 2827 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s user 0.000s sys 0.011s
W20260619 21:16:12.759595 2827 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s user 0.000s sys 0.011s
I20260619 21:16:12.759279 2856 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.527770 2810 consensus_queue.cc:1046] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.234s
W20260619 21:16:12.601071 2836 log.cc:924] Time spent T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.098s user 0.000s sys 0.001s
I20260619 21:16:12.548266 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.548111 2697 consensus_queue.cc:237] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.765625 1668 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:12.765781 1741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 498) took 2463 ms. Trace:
I20260619 21:16:12.766152 1741 rpcz_store.cc:276] 0619 21:16:10.302794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.854734 (+1551940us) service_pool.cc:227] Handling call
0619 21:16:11.912081 (+ 57347us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:11.912332 (+ 251us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:11.952245 (+ 39913us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:11.952286 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:11.952900 (+ 614us) log.cc:842] Serialized 26 byte log entry
0619 21:16:12.285951 (+333051us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:12.301291 (+ 15340us) log.cc:842] Serialized 14 byte log entry
0619 21:16:12.301556 (+ 265us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.301627 (+ 71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:12.466844 (+165217us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b180030acc0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb96638 00007f719bb99b11 00007f719bb99ac9 00007f719bb99a59 00007f719bb99a21 00007f719bb98d1d 00007f71a4dfa204 00007f71a4dfa039 00007f719bb78dc6
0619 21:16:12.765593 (+298749us) raft_consensus.cc:1671] finished
0619 21:16:12.765612 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.765724 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37358,"prepare.queue_time_us":37040,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":34945,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":18872,"spinlock_wait_cycles":76686464,"thread_start_us":331699,"threads_started":1,"wal-append.queue_time_us":407923}
I20260619 21:16:12.637537 1708 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:12.668097 2683 consensus_meta.cc:287] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.566s user 0.009s sys 0.002s
W20260619 21:16:12.559667 2832 log.cc:924] Time spent T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.093s user 0.000s sys 0.000s
W20260619 21:16:12.768323 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 130875us
I20260619 21:16:12.746407 1666 rpcz_store.cc:276] 0619 21:16:11.649682 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.652271 (+ 2589us) service_pool.cc:227] Handling call
0619 21:16:11.655887 (+ 3616us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:11.776775 (+120888us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:11.821671 (+ 44896us) log.cc:842] Serialized 14 byte log entry
0619 21:16:12.337309 (+515638us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.349439 (+ 12130us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.353024 (+ 3585us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.666556 (+313532us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400150754. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:12.666611 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.666752 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4860,"prepare.queue_time_us":23299,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":9762,"raft.queue_time_us":27279,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":12213,"spinlock_wait_cycles":233722240,"thread_start_us":455560,"threads_started":1,"wal-append.queue_time_us":776149}
I20260619 21:16:12.768642 2683 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:12.634172 2772 consensus_meta.cc:287] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.540s user 0.010s sys 0.005s
I20260619 21:16:12.769734 1708 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:12.769981 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 231516us
I20260619 21:16:12.761504 2856 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:12.589231 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:12.580813 2673 consensus_queue.cc:1035] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.182s
I20260619 21:16:12.613698 2759 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:12.775681 2759 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:16:12.775728 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 348702us
W20260619 21:16:12.761276 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 1764e19eb06140f8bbdb1c3aab252d9a (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 21131e98f1f94e8cac1998e6c32aa089
I20260619 21:16:12.776481 1782 leader_election.cc:302] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.777483 2856 raft_consensus.cc:3037] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
W20260619 21:16:12.777985 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 5ae5f9625019495d85bc13c82d7f621b (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 6ae495cfaaca419c8611fe233762a02a
W20260619 21:16:12.779335 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet e3b2adcb38244b60bdd7ae5d5642f641 (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 18756ff2e1e84eb5b7139ecabf689d6a
W20260619 21:16:12.780508 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 0dea6228e49c44b9889aa709158ef557 (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 84b6bbbb60204347824250a8d71e6d3c
I20260619 21:16:12.606616 1737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 520) took 1893 ms. Trace:
W20260619 21:16:12.666947 1735 consensus_meta.cc:287] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.606s user 0.014s sys 0.003s
I20260619 21:16:12.658543 1860 raft_consensus.cc:348] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.786190 1860 raft_consensus.cc:374] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:12.786525 1860 raft_consensus.cc:729] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:12.763574 1661 raft_consensus.cc:2354] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:12.560560 2754 raft_consensus.cc:686] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:16:12.660152 1812 consensus_meta.cc:287] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.546s user 0.008s sys 0.003s
I20260619 21:16:12.797242 1812 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:12.532727 1635 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:12.656978 2775 raft_consensus.cc:686] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:12.770747 2683 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.775249 2716 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:12.780650 2693 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:12.609014 2421 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.814989 1744 raft_consensus.cc:1261] T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260619 21:16:12.816092 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 560) took 1395 ms. Trace:
I20260619 21:16:12.816583 1812 rpcz_store.cc:276] 0619 21:16:11.420275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.866850 (+446575us) service_pool.cc:227] Handling call
0619 21:16:12.815874 (+949024us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b54001162d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:12.816015 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":80926848}
I20260619 21:16:12.741269 1811 rpcz_store.cc:276] 0619 21:16:11.400106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.933287 (+533181us) service_pool.cc:227] Handling call
0619 21:16:11.933638 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.378814 (+445176us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.665131 (+286317us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400134554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:12.665358 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":628047104}
I20260619 21:16:12.735958 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:12.668313 2726 consensus_meta.cc:287] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.567s user 0.009s sys 0.002s
I20260619 21:16:12.785756 1667 raft_consensus.cc:2454] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
W20260619 21:16:12.822034 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 9ae81d060b714ae98341d6b5e2669f3f (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 331f6581b0ac4c3bbab86ecc34ececfb
I20260619 21:16:12.822938 1667 raft_consensus.cc:1226] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
W20260619 21:16:12.822999 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 290336us
I20260619 21:16:12.787264 1860 consensus_queue.cc:260] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.822043 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.791045 1782 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.793398 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.772615 1634 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.796178 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.772284 1708 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:12.711954 1695 tablet.cc:2369] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 21:16:12.827421 1739 raft_consensus.cc:2379] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:16:12.827706 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 55490.6us
I20260619 21:16:12.828217 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:12.828363 1708 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.803663 2602 consensus_queue.cc:1035] T 0fe8c95af50740a783386324c1649be0 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:12.804554 1783 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.808583 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:12.839800 2726 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.840550 1743 raft_consensus.cc:2379] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:12.785282 1735 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:12.841364 1735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 538) took 1152 ms. Trace:
I20260619 21:16:12.841593 1740 raft_consensus.cc:1261] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:12.842049 2726 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.784628 2856 raft_consensus.cc:2726] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:12.842559 2764 raft_consensus.cc:2726] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:12.843505 2754 consensus_queue.cc:237] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.844359 2775 consensus_queue.cc:237] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.846180 2810 consensus_queue.cc:1225] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer b2e7f02778ec43bdbca03dd013687c76 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260619 21:16:12.785759 2719 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:12.819275 1812 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 5. Current term is 6. Ops: []
I20260619 21:16:12.814117 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:12.821460 1668 raft_consensus.cc:1226] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:16:12.838260 2683 raft_consensus.cc:482] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:12.848361 1635 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:12.849511 1708 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.855940 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.858385 1708 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.850499 1663 raft_consensus.cc:2454] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:12.852401 2684 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:12.861581 2460 raft_consensus.cc:2726] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:12.861901 2684 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:12.862401 2684 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:12.862560 2716 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:12.857191 2422 consensus_queue.cc:1046] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260619 21:16:12.864713 2422 raft_consensus.cc:3032] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:12.865175 2422 raft_consensus.cc:729] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:12.866250 2421 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:12.852665 2625 raft_consensus.cc:2781] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:12.867022 2625 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:12.867415 2625 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:12.867637 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet a7a69751ebea42d5950577710f9b1301
I20260619 21:16:12.869045 2683 raft_consensus.cc:504] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.869639 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:16:12.868793 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 246c3d015ec242eca8bfaf5a076692b1 (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 655f9756924a470b8654421a17c9322e
W20260619 21:16:12.870949 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 66510.8us
W20260619 21:16:12.872907 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 100351us
I20260619 21:16:12.873646 1634 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.874480 1815 raft_consensus.cc:2379] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:12.841753 1735 rpcz_store.cc:276] 0619 21:16:11.689451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:11.825256 (+135805us) service_pool.cc:227] Handling call
0619 21:16:12.841306 (+1016050us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:12.769752 2772 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.883584 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.881687 1708 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.882550 1634 leader_election.cc:302] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.886229 1708 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.888902 1860 ts_tablet_manager.cc:1428] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.406s user 0.005s sys 0.006s
I20260619 21:16:12.894924 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.894203 1708 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.894994 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:12.924926 1814 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:12.878219 2809 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260619 21:16:12.929669 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 138702us
I20260619 21:16:12.882488 2847 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
W20260619 21:16:12.927006 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 54579.7us
I20260619 21:16:12.886127 2827 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:12.887370 2719 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:12.934571 1782 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.887554 2693 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:12.970928 1708 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.870389 2784 consensus_queue.cc:1035] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260619 21:16:12.886556 2716 raft_consensus.cc:2726] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:12.912734 2764 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260619 21:16:12.976018 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 43797.7us
I20260619 21:16:12.976336 1708 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:12.926674 2726 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:12.988193 1663 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 5. Current term is 6. Ops: []
I20260619 21:16:12.992741 1708 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:12.994055 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 59556.7us
I20260619 21:16:12.980688 1809 raft_consensus.cc:1226] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260619 21:16:12.994760 1782 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:12.889748 2759 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:12.915361 2421 consensus_queue.cc:1046] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260619 21:16:12.929919 2810 consensus_queue.cc:1046] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260619 21:16:12.989908 2719 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:13.027201 1708 leader_election.cc:398] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3.
I20260619 21:16:13.029968 1708 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:13.026012 2871 log.cc:924] Time spent T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.167s user 0.001s sys 0.000s
I20260619 21:16:12.995476 2847 raft_consensus.cc:686] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:13.035663 1815 raft_consensus.cc:1226] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:13.035881 1708 leader_election.cc:302] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:12.998589 2797 raft_consensus.cc:2781] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:12.985114 2860 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:13.033278 2858 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:12.997589 2861 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:13.039417 2797 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:13.064309 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:12.998821 2771 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:12.991712 2795 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260619 21:16:13.093778 1663 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:13.038539 2775 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:13.037567 2774 consensus_queue.cc:1046] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260619 21:16:13.052740 2683 leader_election.cc:288] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.059866 2797 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:13.106911 2784 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:13.107483 2784 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:13.108393 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet cfa69d158f2b471e8638824d3d9c757b (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 7f05cdc19f4e420880bab15773a59709
W20260619 21:16:13.109550 2874 log.cc:924] Time spent T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20260619 21:16:13.109628 2784 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.111541 1662 raft_consensus.cc:1226] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:13.111163 2422 consensus_queue.cc:260] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.112236 2422 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:13.112877 2764 raft_consensus.cc:3032] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:13.060389 2719 raft_consensus.cc:2781] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:13.113267 2764 raft_consensus.cc:729] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:13.105268 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:13.106156 2759 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:13.106467 2869 log.cc:924] Time spent T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.107s user 0.001s sys 0.000s
I20260619 21:16:13.113517 2719 raft_consensus.cc:686] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:13.130471 1860 tablet_bootstrap.cc:492] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:12.785157 1737 rpcz_store.cc:276] 0619 21:16:10.712853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.095105 (+1382252us) service_pool.cc:227] Handling call
0619 21:16:12.095404 (+ 299us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:12.278102 (+182698us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:12.334636 (+ 56534us) log.cc:842] Serialized 14 byte log entry
0619 21:16:12.486559 (+151923us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:12.503513 (+ 16954us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:12.505781 (+ 2268us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:12.606320 (+100539us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540026eb54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:12.606368 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:12.606527 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12409,"prepare.queue_time_us":46865,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":7483,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9343,"spinlock_wait_cycles":383404416,"thread_start_us":123428,"threads_started":1,"wal-append.queue_time_us":191631}
I20260619 21:16:13.059940 2726 consensus_queue.cc:1046] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.071s
W20260619 21:16:13.134645 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 252151us
I20260619 21:16:13.132005 1665 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:13.127879 2772 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:13.061817 2861 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:13.145458 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:13.137355 2784 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:13.147423 2872 log.cc:924] Time spent T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.094s user 0.000s sys 0.000s
I20260619 21:16:13.147058 2797 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.148077 2719 consensus_queue.cc:237] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.139496 2764 consensus_queue.cc:260] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.150015 2764 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:13.151489 1663 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
W20260619 21:16:13.153714 2870 log.cc:924] Time spent T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.185s user 0.001s sys 0.000s
I20260619 21:16:13.143420 2788 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:13.157609 2788 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.158102 2625 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.159135 1741 raft_consensus.cc:1226] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:13.159737 2788 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.146971 2784 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.145730 2861 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:13.199263 1664 raft_consensus.cc:2454] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:13.134063 2847 consensus_queue.cc:237] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.202435 1661 raft_consensus.cc:3032] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:13.202884 1661 raft_consensus.cc:729] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:13.203678 1661 consensus_queue.cc:260] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.207080 2784 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.204864 1661 raft_consensus.cc:3037] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:13.218268 2625 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.195933 2684 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
@ 0x7f719b22b9a0 google::LogMessage::Flush() at ??:0
I20260619 21:16:13.223206 1860 tablet_bootstrap.cc:654] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:13.223558 2797 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:13.234261 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 577) took 1186 ms. Trace:
I20260619 21:16:13.237803 1814 rpcz_store.cc:276] 0619 21:16:12.047861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.053342 (+ 5481us) service_pool.cc:227] Handling call
0619 21:16:13.233988 (+1180646us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400279554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:13.234185 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1860758784}
I20260619 21:16:13.239406 2809 raft_consensus.cc:2726] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:13.243386 2827 raft_consensus.cc:686] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:13.246270 1665 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
W20260619 21:16:13.253057 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 115986us
I20260619 21:16:13.255106 1708 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:13.256170 2697 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:13.256817 2697 raft_consensus.cc:686] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:13.257917 2697 consensus_queue.cc:237] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.268901 1667 raft_consensus.cc:1226] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 2. Current term is 3. Ops: []
W20260619 21:16:13.273298 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 133350us
I20260619 21:16:13.274111 1634 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:13.277876 1634 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:13.270742 2693 raft_consensus.cc:3032] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Stepping down as leader of term 2
W20260619 21:16:13.301710 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 311985us
W20260619 21:16:13.301754 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 307029us
I20260619 21:16:13.307716 1782 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:13.309000 1708 leader_election.cc:302] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:13.244601 2827 consensus_queue.cc:237] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.307685 2856 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:13.316205 2809 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:13.332410 2856 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:13.318374 1661 raft_consensus.cc:1261] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:13.338172 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:13.307376 2606 consensus_queue.cc:1035] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260619 21:16:13.340026 2805 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260619 21:16:13.344669 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:13.307251 2885 consensus_queue.cc:1046] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260619 21:16:13.347409 1816 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:13.353503 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:13.316246 2886 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:13.329604 2713 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:13.359508 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:13.360297 2713 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.364194 2713 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:13.314275 1783 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:13.335565 1708 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:13.333079 2856 raft_consensus.cc:3037] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:13.344494 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:13.301445 2693 raft_consensus.cc:729] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:13.309064 2860 raft_consensus.cc:686] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:16:13.348165 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 70362.3us
I20260619 21:16:13.356525 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:13.373874 1669 raft_consensus.cc:1261] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:13.357077 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:13.359040 2810 raft_consensus.cc:3032] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:13.330680 2684 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.358469 2683 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260619 21:16:13.374399 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet b249d6b40f6943e3bb7235b7df99e1b1 (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 016af7231f1345d3b13158794ac82632
I20260619 21:16:13.319433 2887 raft_consensus.cc:2781] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election won for term 1
W20260619 21:16:13.382598 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 74976us
I20260619 21:16:13.392190 1710 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:13.393122 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 78937.4us
I20260619 21:16:13.393227 2865 consensus_queue.cc:1046] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:13.394260 2865 consensus_queue.cc:1046] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:13.395314 2623 raft_consensus.cc:2781] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:13.395574 2774 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:13.395830 2623 raft_consensus.cc:686] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:13.396763 2623 consensus_queue.cc:237] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.398705 2883 consensus_queue.cc:1035] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:13.400904 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:13.332701 2809 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.404808 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:13.407421 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:13.400292 2527 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:13.411772 1782 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:13.422222 1742 raft_consensus.cc:1226] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:13.424278 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:13.425067 1740 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:13.390452 2861 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.380963 2810 raft_consensus.cc:729] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:13.436388 1634 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:13.411489 2887 raft_consensus.cc:686] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:13.423533 2883 consensus_queue.cc:1035] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260619 21:16:13.438937 2887 consensus_queue.cc:237] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.442102 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:13.443096 2858 raft_consensus.cc:3032] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:13.443624 2858 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:13.445943 2684 consensus_queue.cc:1035] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:13.404469 2859 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:13.349911 2422 consensus_queue.cc:1046] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260619 21:16:13.397745 2693 consensus_queue.cc:260] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.429502 2885 raft_consensus.cc:3032] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Stepping down as leader of term 4
W20260619 21:16:13.484405 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:13.488524 1738 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:13.450039 2859 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:13.462075 2847 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:13.449373 2734 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:13.454457 2527 raft_consensus.cc:686] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:13.467674 2716 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:13.496876 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 55627.5us
W20260619 21:16:13.504456 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 109932us
W20260619 21:16:13.515219 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 68379.1us
I20260619 21:16:13.488972 2684 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:13.473178 2885 raft_consensus.cc:729] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:13.491310 2422 consensus_queue.cc:1225] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer 884c214bb0a7455d91701ec388635e74 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260619 21:16:13.525866 1739 raft_consensus.cc:1261] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:13.512975 2827 raft_consensus.cc:2741] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260619 21:16:13.498713 2847 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:13.583470 2795 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:13.581470 1740 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:13.513907 2734 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.517880 2716 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:13.612975 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 78961.4us
I20260619 21:16:13.472049 2806 consensus_queue.cc:1046] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260619 21:16:13.532135 2865 consensus_queue.cc:1046] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260619 21:16:13.509200 2859 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:13.581605 2684 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.444540 2858 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.515411 2693 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:13.612411 2422 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:16:13.662355 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 138724us
I20260619 21:16:13.663244 1710 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:13.651690 2847 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:13.618968 2774 consensus_queue.cc:1046] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260619 21:16:13.620543 2606 consensus_queue.cc:1035] T 692d38d1f19c468eae7077dde218a62f P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260619 21:16:13.498850 2810 consensus_queue.cc:260] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.668828 2806 consensus_queue.cc:1225] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer b2e7f02778ec43bdbca03dd013687c76 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260619 21:16:13.658320 2795 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:13.706847 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 52678.8us
I20260619 21:16:13.692555 2827 raft_consensus.cc:3032] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:13.496062 2860 consensus_queue.cc:237] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.666863 2422 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:13.734807 2861 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:13.706157 2904 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:13.666214 2527 consensus_queue.cc:237] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:13.811746 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 95112.5us
I20260619 21:16:13.714731 2827 raft_consensus.cc:729] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:13.702607 2809 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:13.844097 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 180938us
W20260619 21:16:13.844192 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet d80394dabe604f8898e7b24149fc624f (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet ae65b717a68f436586f575875021805b
W20260619 21:16:13.813148 2888 log.cc:924] Time spent T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.342s user 0.000s sys 0.001s
I20260619 21:16:13.833781 2771 raft_consensus.cc:2781] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:13.715752 2673 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:13.833353 2810 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
W20260619 21:16:13.926416 2877 log.cc:924] Time spent T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.811s user 0.005s sys 0.002s
I20260619 21:16:13.664069 2885 consensus_queue.cc:260] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.030254 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 588) took 1220 ms. Trace:
W20260619 21:16:13.953871 2911 log.cc:924] Time spent T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.152s user 0.000s sys 0.000s
W20260619 21:16:13.963160 2910 log.cc:924] Time spent T 8f709f3b1e904d1ba8bf5592d77cc8d6 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.136s user 0.000s sys 0.000s
W20260619 21:16:13.968050 2896 log.cc:924] Time spent T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.341s user 0.000s sys 0.001s
W20260619 21:16:13.896587 2871 log.cc:924] Time spent T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.094s user 0.001s sys 0.000s
I20260619 21:16:13.833738 2904 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:13.929771 2878 log.cc:924] Time spent T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.092s user 0.001s sys 0.000s
I20260619 21:16:13.993112 2827 consensus_queue.cc:260] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:13.898321 2879 log.cc:924] Time spent T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.096s user 0.000s sys 0.001s
I20260619 21:16:13.832219 2919 consensus_queue.cc:1046] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
I20260619 21:16:13.899842 2606 consensus_queue.cc:1035] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.098s
W20260619 21:16:14.024168 1782 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.037s sys 0.033s
W20260619 21:16:13.914002 2856 consensus_meta.cc:287] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.522s user 0.010s sys 0.002s
I20260619 21:16:13.901551 2734 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.030869 1810 rpcz_store.cc:276] 0619 21:16:12.809453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.809833 (+ 380us) service_pool.cc:227] Handling call
0619 21:16:12.810247 (+ 414us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:12.876705 (+ 66458us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:12.876734 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:12.877151 (+ 417us) log.cc:842] Serialized 26 byte log entry
0619 21:16:13.273871 (+396720us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:16:13.273909 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:13.364411 (+ 90502us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b54001171d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:13.364474 (+ 63us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:14.029859 (+665385us) raft_consensus.cc:1671] finished
0619 21:16:14.029887 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:14.030171 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":99054,"raft.queue_time_us":33390,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":1403,"spinlock_wait_cycles":88780800,"thread_start_us":396405,"threads_started":1,"wal-append.queue_time_us":531570}
I20260619 21:16:14.056751 2602 consensus_queue.cc:1035] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260619 21:16:13.992472 2673 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:14.111258 1782 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.593s user 0.040s sys 0.036s
W20260619 21:16:14.122165 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 605898us
I20260619 21:16:14.126596 2919 consensus_queue.cc:1225] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer d53146739e3240239c0a3d5ea90e07fe log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260619 21:16:14.127305 2602 raft_consensus.cc:3032] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:14.127842 2602 raft_consensus.cc:729] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:14.022586 2771 raft_consensus.cc:686] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:14.132540 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.133909 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.080832 2827 raft_consensus.cc:3037] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:14.073333 2684 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.135525 1782 leader_election.cc:302] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.138964 2606 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader election won for term 6
I20260619 21:16:14.151840 1782 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.154779 1782 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.134662 2716 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.160313 2919 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.155378 2754 raft_consensus.cc:2781] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:14.179698 2754 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:14.179500 2919 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.180209 2754 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:14.181972 2919 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.182497 2606 raft_consensus.cc:686] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:14.183516 2606 consensus_queue.cc:237] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.151623 2602 consensus_queue.cc:260] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.185336 2795 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.185879 2795 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:14.185979 2602 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:14.186235 2795 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:14.186597 2795 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:14.160889 2772 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:14.130183 2856 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.156553 2672 raft_consensus.cc:2726] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:14.193615 2422 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.240865 2672 consensus_queue.cc:1035] T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:14.254928 2921 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.261929 2921 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.264155 2921 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.253134 2919 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:14.265614 2919 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.266165 2754 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:14.265146 2917 log.cc:924] Time spent T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260619 21:16:14.267560 2904 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:14.331387 2693 consensus_meta.cc:287] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.666s user 0.011s sys 0.004s
I20260619 21:16:14.333148 2921 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:14.335312 2683 raft_consensus.cc:3032] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:14.332553 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:14.336716 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet 692d38d1f19c468eae7077dde218a62f (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 35cdbd73611d4dcd915a0971b48e3d49
W20260619 21:16:14.337061 2859 consensus_meta.cc:287] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.669s user 0.012s sys 0.000s
I20260619 21:16:14.211287 2771 consensus_queue.cc:237] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.342692 2683 raft_consensus.cc:729] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
W20260619 21:16:14.362285 1814 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.749s user 0.010s sys 0.021s
W20260619 21:16:14.362777 1814 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.750s user 0.010s sys 0.021s
I20260619 21:16:14.349037 2673 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:14.373313 1634 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.922s user 0.021s sys 0.037s
W20260619 21:16:14.373842 1634 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.924s user 0.021s sys 0.037s
W20260619 21:16:14.374562 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 938103us
I20260619 21:16:14.375545 1634 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.348464 2921 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:14.368582 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 523366us
W20260619 21:16:14.373070 1783 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.533s user 0.010s sys 0.053s
W20260619 21:16:14.379916 1783 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.540s user 0.010s sys 0.053s
W20260619 21:16:14.380388 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 549532us
I20260619 21:16:14.381250 1783 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:14.379536 1710 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:14.383946 1708 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.854s user 0.011s sys 0.026s
W20260619 21:16:14.384339 1708 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.855s user 0.011s sys 0.026s
I20260619 21:16:14.350298 2859 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.379263 2856 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:14.388305 2797 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s user 0.006s sys 0.031s
I20260619 21:16:14.398173 1667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 531) took 1935 ms. Trace:
I20260619 21:16:14.398521 1667 rpcz_store.cc:276] 0619 21:16:12.462411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.823672 (+361261us) service_pool.cc:227] Handling call
0619 21:16:12.874987 (+ 51315us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:13.269421 (+394434us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:14.397913 (+1128492us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400332f54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:14.398086 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":905134464}
I20260619 21:16:14.400028 1741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 552) took 1608 ms. Trace:
I20260619 21:16:14.400326 1741 rpcz_store.cc:276] 0619 21:16:12.791177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:12.797416 (+ 6239us) service_pool.cc:227] Handling call
0619 21:16:12.854608 (+ 57192us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ebce 000000000034aea9 00007f71a398aa30 00007f71a3a507b4 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:12.854818 (+ 210us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:13.159473 (+304655us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:14.399843 (+1240370us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54001a2554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:14.399982 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":704402816}
W20260619 21:16:14.395090 2797 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s user 0.006s sys 0.031s
I20260619 21:16:14.397771 2856 raft_consensus.cc:2781] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 21:16:14.391955 2422 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.401849 2919 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.402391 2922 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.407641 1710 leader_election.cc:302] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.384466 2683 consensus_queue.cc:260] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:14.383363 2623 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s user 0.015s sys 0.019s
I20260619 21:16:14.410923 2931 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:14.414506 2719 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:14.422641 2754 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:14.439735 1709 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.502s user 0.004s sys 0.008s
W20260619 21:16:14.428972 2623 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s user 0.016s sys 0.019s
I20260619 21:16:14.444732 1710 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:14.416045 2422 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.449961 1710 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.434126 2764 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.453513 2764 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.459731 2893 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:14.467623 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 92163.1us
I20260619 21:16:14.468655 1634 leader_election.cc:302] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.471184 1634 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.435577 2929 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.473227 1634 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.437451 2856 raft_consensus.cc:686] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
W20260619 21:16:14.473234 1668 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s user 0.037s sys 0.069s
W20260619 21:16:14.474560 1668 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s user 0.037s sys 0.069s
I20260619 21:16:14.460289 2893 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.445242 2683 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:14.450793 2716 raft_consensus.cc:2726] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:14.490679 1634 leader_election.cc:398] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 5. Current term is 6.
I20260619 21:16:14.491664 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:14.492427 1669 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:14.492604 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.505903 1665 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
W20260619 21:16:14.481956 2847 consensus_meta.cc:287] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.796s user 0.016s sys 0.000s
W20260619 21:16:14.447924 2883 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.742s user 0.012s sys 0.046s
W20260619 21:16:14.507723 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 57893.2us
I20260619 21:16:14.476123 2900 raft_consensus.cc:2741] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260619 21:16:14.472923 2865 raft_consensus.cc:2726] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:14.513759 1860 tablet_bootstrap.cc:492] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:14.514192 1710 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.516166 2774 raft_consensus.cc:2726] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:14.482522 2684 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260619 21:16:14.520107 2883 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.805s user 0.013s sys 0.047s
I20260619 21:16:14.520459 2847 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.477773 2795 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.521755 1634 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:14.565081 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 50985.4us
W20260619 21:16:14.565578 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 43908.7us
I20260619 21:16:14.490774 2806 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260619 21:16:14.528337 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:14.525784 1860 ts_tablet_manager.cc:1397] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.396s user 0.018s sys 0.027s
W20260619 21:16:14.542929 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 67505.2us
I20260619 21:16:14.473593 2930 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:14.427554 2922 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.495289 2810 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.435949 2931 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.522783 2921 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.597241 1634 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.542146 2684 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.487303 2929 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.549590 2764 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.643401 1810 raft_consensus.cc:1261] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:14.599254 2923 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.542340 2460 consensus_queue.cc:1035] T bb9b500bf9a24148b2fbcd762b549521 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260619 21:16:14.595989 2865 raft_consensus.cc:2726] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:14.507781 2924 consensus_queue.cc:1046] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
I20260619 21:16:14.513752 2716 consensus_queue.cc:1046] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260619 21:16:14.648515 1813 raft_consensus.cc:1226] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:14.649925 2893 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.548420 2859 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.632022 2810 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.650538 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:14.651036 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.651381 1809 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:14.620762 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 567) took 1220 ms. Trace:
I20260619 21:16:14.634416 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.619869 2856 consensus_queue.cc:237] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.652264 2810 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.653009 1662 raft_consensus.cc:2396] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:14.652117 1738 rpcz_store.cc:276] 0619 21:16:13.400515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:13.400647 (+ 132us) service_pool.cc:227] Handling call
0619 21:16:14.617979 (+1217332us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b540022c754. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:14.620683 (+ 2704us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":195468032}
I20260619 21:16:14.654366 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.654548 1813 raft_consensus.cc:1226] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:14.654749 1816 raft_consensus.cc:1261] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:14.655126 1738 raft_consensus.cc:2379] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:14.634160 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.655544 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.656006 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:14.656359 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:14.635501 2931 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.657167 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.657394 2902 raft_consensus.cc:3032] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:14.657644 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:14.657845 2902 raft_consensus.cc:729] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:14.658710 2902 consensus_queue.cc:260] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.649138 2923 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.660007 2902 raft_consensus.cc:3037] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:14.660151 1814 raft_consensus.cc:1261] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:14.661161 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.634146 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.635655 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.636780 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.636955 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.637898 1634 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.638866 2922 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.639441 2805 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260619 21:16:14.622577 2913 log.cc:924] Time spent T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260619 21:16:14.566407 2883 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260619 21:16:14.631587 2891 log.cc:924] Time spent T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 1.221s user 0.011s sys 0.000s
I20260619 21:16:14.664420 1739 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:14.646335 1810 raft_consensus.cc:1261] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260619 21:16:14.619690 2602 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.629007 2930 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:14.670755 1743 raft_consensus.cc:1261] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:14.647272 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:14.650029 2938 log.cc:924] Time spent T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260619 21:16:14.650635 2847 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.636093 1664 raft_consensus.cc:1261] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:14.631184 1669 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:14.647883 2774 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.666247 1810 pending_rounds.cc:76] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Aborting all ops after (but not including) 1
I20260619 21:16:14.676738 1810 pending_rounds.cc:97] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260619 21:16:14.677161 1810 raft_consensus.cc:2866] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260619 21:16:14.669471 2827 consensus_meta.cc:287] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.520s user 0.012s sys 0.000s
I20260619 21:16:14.668566 1634 leader_election.cc:302] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.681005 1634 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.646869 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.664991 1738 raft_consensus.cc:2379] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:14.640815 2684 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
@ 0x7f719b230adb google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260619 21:16:14.682324 1634 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.685915 2805 raft_consensus.cc:2726] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:14.671700 2810 raft_consensus.cc:2781] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:14.757314 1634 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.760124 2858 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:14.762853 1860 raft_consensus.cc:348] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.785956 1860 raft_consensus.cc:374] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:14.786305 1860 raft_consensus.cc:729] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:14.673087 2764 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:14.764492 2775 raft_consensus.cc:3032] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:14.675971 2923 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.673316 2930 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:14.691021 2422 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.681530 2774 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.774062 2827 raft_consensus.cc:2781] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260619 21:16:14.689841 2934 log.cc:924] Time spent T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
W20260619 21:16:14.776638 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 96170.2us
W20260619 21:16:14.776386 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 110240us
W20260619 21:16:14.782553 1635 consensus_peers.cc:487] T 9ae81d060b714ae98341d6b5e2669f3f P 884c214bb0a7455d91701ec388635e74 -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260619 21:16:14.782429 1634 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.780198 2810 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:14.691380 2876 raft_consensus.cc:2726] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:14.672856 2602 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.833154 1817 raft_consensus.cc:1226] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 5. Current term is 6. Ops: []
W20260619 21:16:14.862054 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 79721.3us
I20260619 21:16:14.878985 1634 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.879532 1708 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:14.879663 1783 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:14.850173 1621 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Tablet bb9b500bf9a24148b2fbcd762b549521 (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) was not created within the allowed timeout. Replacing with a new tablet 96effdfbfd2640ce8ee752e99d252d9e
I20260619 21:16:14.832494 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.880533 1634 leader_election.cc:302] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:14.841656 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 51735.6us
W20260619 21:16:14.863456 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.33858e+06us
I20260619 21:16:14.787176 1860 consensus_queue.cc:260] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.790867 1815 raft_consensus.cc:1226] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:14.779131 2527 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.885393 1634 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.788568 2795 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.784369 2933 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.797394 2805 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:14.903481 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.887266 1860 ts_tablet_manager.cc:1428] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.291s user 0.004s sys 0.004s
I20260619 21:16:14.905231 1812 raft_consensus.cc:1226] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:16:14.905323 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.907883 2625 consensus_queue.cc:1046] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:14.908104 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.877302 2810 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:14.908828 2625 consensus_queue.cc:1035] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:14.909693 2784 raft_consensus.cc:3032] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:14.910035 2784 raft_consensus.cc:729] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:14.910810 2784 consensus_queue.cc:260] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.912112 2784 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:14.914391 2602 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.913928 1710 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:14.833652 1735 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:14.915436 2422 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:14.916304 2719 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:14.916460 2422 consensus_queue.cc:1035] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:14.917032 2719 raft_consensus.cc:686] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:14.918146 2719 consensus_queue.cc:237] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.847572 2885 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:14.877813 2809 raft_consensus.cc:2726] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:14.865851 2827 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:14.898289 2806 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:14.903347 2887 consensus_queue.cc:1046] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:14.905689 1860 tablet_bootstrap.cc:492] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:14.861785 2775 raft_consensus.cc:729] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:14.922806 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.845481 2726 consensus_queue.cc:1035] T 516fa114155140f6b508f311c4971c08 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260619 21:16:14.885752 2876 raft_consensus.cc:2726] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:14.924297 1663 raft_consensus.cc:1261] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:14.877373 2764 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.942763 2809 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:14.932338 2805 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.948099 2809 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.948170 2805 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.932598 2693 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:14.931030 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.897357 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.933930 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:14.938138 1666 raft_consensus.cc:1261] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:14.881170 2893 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:14.945053 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.845041 2772 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:14.963241 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.961728 2421 raft_consensus.cc:2726] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:14.963925 1666 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:14.964979 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:14.963189 2772 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.966157 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.967213 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.898733 2754 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:14.941210 1783 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:14.930220 1813 raft_consensus.cc:1261] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:14.938899 2827 raft_consensus.cc:3037] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:14.939600 1669 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:14.973977 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:14.958001 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 60798.2us
I20260619 21:16:14.960737 2859 raft_consensus.cc:3032] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:14.961031 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.920214 2933 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.961687 2775 consensus_queue.cc:260] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.962167 2887 consensus_queue.cc:1035] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:14.967284 2772 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:14.967764 2754 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:14.920238 2810 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:14.941903 2771 consensus_queue.cc:1035] T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260619 21:16:14.972445 1739 raft_consensus.cc:1261] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:15.030942 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.032166 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.942301 2865 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:15.034734 1660 raft_consensus.cc:2379] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:15.032972 2422 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:15.034613 2865 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.035429 2422 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.035765 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:14.976052 1634 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:15.038645 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 62667.3us
I20260619 21:16:15.039372 1634 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:14.944576 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.974596 1669 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:14.977690 2859 raft_consensus.cc:729] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:14.885030 2935 consensus_queue.cc:1035] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.061s
I20260619 21:16:14.980034 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:14.940224 2460 consensus_queue.cc:1035] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260619 21:16:14.980867 2805 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.981451 1741 raft_consensus.cc:1261] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260619 21:16:15.029619 2754 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:15.052292 1741 pending_rounds.cc:76] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: Aborting all ops after (but not including) 1
I20260619 21:16:15.052583 1741 pending_rounds.cc:97] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260619 21:16:14.981657 2764 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.941087 2923 consensus_queue.cc:1035] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260619 21:16:14.983122 2933 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.974401 2922 consensus_queue.cc:1035] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260619 21:16:14.983783 1666 raft_consensus.cc:2379] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:15.056591 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.057262 1666 raft_consensus.cc:2379] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:14.983999 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:14.986299 2883 raft_consensus.cc:3032] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:14.977929 1708 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:14.950091 1668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 556) took 1432 ms. Trace:
I20260619 21:16:14.970157 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.902443 2527 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.042173 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.061388 1669 raft_consensus.cc:2379] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:15.062047 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.062610 1669 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:15.063409 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:14.977487 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:14.968684 2693 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.050642 1634 leader_election.cc:302] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.052817 1741 raft_consensus.cc:2866] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:15.046761 2775 raft_consensus.cc:3037] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:14.987987 2929 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:14.987752 2925 raft_consensus.cc:2781] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:15.058650 1668 rpcz_store.cc:276] 0619 21:16:13.517401 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:13.517524 (+ 123us) service_pool.cc:227] Handling call
0619 21:16:13.617314 (+ 99790us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:13.617880 (+ 566us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:13.617928 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:13.627271 (+ 9343us) log.cc:842] Serialized 26 byte log entry
0619 21:16:14.474993 (+847722us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:14.475040 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:14.475121 (+ 81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:14.949828 (+474707us) raft_consensus.cc:1671] finished
0619 21:16:14.949856 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:14.950016 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11608,"raft.queue_time_us":173,"raft.run_cpu_time_us":424,"raft.run_wall_time_us":34314,"thread_start_us":810999,"threads_started":1,"wal-append.queue_time_us":890070}
I20260619 21:16:15.089151 2933 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:15.025084 2893 consensus_queue.cc:1046] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260619 21:16:15.092094 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 114231us
I20260619 21:16:15.046607 2602 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:15.042822 2865 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.051923 2806 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:15.089480 1664 raft_consensus.cc:2379] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:15.055660 2772 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:15.053910 2900 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:15.103561 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.108361 2527 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.061684 2883 raft_consensus.cc:729] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
W20260619 21:16:15.089488 2948 log.cc:924] Time spent T 4b1e21185de74e3da01e18ff99572a08 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.121s user 0.000s sys 0.000s
I20260619 21:16:15.117214 1666 raft_consensus.cc:1261] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260619 21:16:15.124068 1666 pending_rounds.cc:76] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Aborting all ops after (but not including) 2
I20260619 21:16:15.104425 2925 raft_consensus.cc:686] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:15.089690 2805 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:15.114154 1668 raft_consensus.cc:1226] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:15.131731 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.132689 1668 raft_consensus.cc:2379] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:15.116323 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.124326 1666 pending_rounds.cc:97] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260619 21:16:15.137666 1666 raft_consensus.cc:2866] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:15.103170 2922 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.102921 2774 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.110721 2602 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.109642 2859 consensus_queue.cc:260] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.094884 2933 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:15.141793 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.143898 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.145943 1663 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:15.146950 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.116689 2772 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.151311 2772 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.153348 2683 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
@ 0x7f71a51f280e kudu::TestWorkload::Setup() at ??:0
I20260619 21:16:15.153903 2683 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
@ 0x345a78 kudu::master::LeaderRebalancerTest::CreateWorkloadTable() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:109
@ 0x3424af kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:411
I20260619 21:16:15.157047 2933 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:15.147765 1668 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:15.116555 2422 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.126807 2883 consensus_queue.cc:260] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.127645 2809 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.160236 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.159281 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.165746 1668 raft_consensus.cc:2379] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:15.167192 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.167966 1668 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:15.168249 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.169009 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.171448 1668 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:15.174484 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.181857 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.164973 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.189935 1664 raft_consensus.cc:2396] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:15.190855 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:16:15.196133 2952 log.cc:924] Time spent T da1270002c3c45988c3ac4fbeafaf019 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.144s user 0.000s sys 0.001s
I20260619 21:16:15.195429 1668 raft_consensus.cc:2454] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:15.197422 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 568) took 1796 ms. Trace:
I20260619 21:16:15.197753 1744 rpcz_store.cc:276] 0619 21:16:13.400813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:13.401564 (+ 751us) service_pool.cc:227] Handling call
0619 21:16:13.401868 (+ 304us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:13.402173 (+ 305us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:13.402200 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:14.664298 (+1262098us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b1800307ea0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb63f96 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719d4a1e11 00007f719d4b1442 00007f719eaf9c11 00007f719eabe58a
0619 21:16:14.761394 (+ 97096us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b2c00000238. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719d4a1e11 00007f719d4b1442 00007f719eaf9c11 00007f719eabe58a 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772
0619 21:16:14.761851 (+ 457us) log.cc:842] Serialized 26 byte log entry
0619 21:16:14.933271 (+171420us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:14.933307 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:14.933387 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:15.197197 (+263810us) raft_consensus.cc:1671] finished
0619 21:16:15.197218 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:15.197363 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8195,"raft.queue_time_us":2065,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":14058,"spinlock_wait_cycles":13407744,"thread_start_us":171046,"threads_started":1,"wal-append.queue_time_us":194595}
I20260619 21:16:15.199321 1661 raft_consensus.cc:1226] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:15.199528 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.200659 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.171624 2859 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:15.201639 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.202240 1744 raft_consensus.cc:2454] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:15.203025 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.203740 1744 raft_consensus.cc:2396] T 0fe8c95af50740a783386324c1649be0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:15.204391 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.205030 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.166850 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.206583 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.145242 2922 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.209326 1783 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.210170 1736 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:15.195734 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.236093 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.236845 1662 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:15.237073 1664 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:15.238054 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.238721 1664 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:15.238822 2672 raft_consensus.cc:2781] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260619 21:16:15.239028 2954 log.cc:924] Time spent T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260619 21:16:15.239199 2672 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:15.239612 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.191082 2925 consensus_queue.cc:237] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.240401 1664 raft_consensus.cc:2379] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:15.239638 2672 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:15.241341 1660 raft_consensus.cc:2454] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:15.242257 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.166491 2923 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260619 21:16:15.242877 1660 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:15.243122 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.243978 1660 raft_consensus.cc:1226] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260619 21:16:15.234381 1668 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:15.258435 1668 raft_consensus.cc:1261] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260619 21:16:15.258775 1668 pending_rounds.cc:76] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Aborting all ops after (but not including) 2
I20260619 21:16:15.258971 1668 pending_rounds.cc:97] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260619 21:16:15.259179 1668 raft_consensus.cc:2866] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:15.238245 1860 tablet_bootstrap.cc:654] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:15.260952 1744 raft_consensus.cc:2379] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 4: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:15.264384 2693 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.249375 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.171563 2860 consensus_queue.cc:1035] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260619 21:16:15.194824 2810 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.267915 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.277251 1744 raft_consensus.cc:2379] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 6: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:15.279623 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.280172 1708 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.281212 2784 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:15.281554 2784 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:15.281847 2784 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:15.282056 2922 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.283208 2827 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.278354 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.163301 2684 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.272262 2859 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.286201 1744 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:15.288254 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.291666 1708 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:15.295238 1742 raft_consensus.cc:2379] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:15.296316 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.297055 1742 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:15.298658 1708 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.317077 2861 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:15.267848 2602 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.333214 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.339872 1634 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.341933 1634 leader_election.cc:302] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.343104 1634 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.344151 1634 leader_election.cc:302] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.346534 1634 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.353946 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.354554 1744 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:15.365602 1634 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.367089 1634 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.337147 2935 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:15.370898 2933 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:15.351147 2858 consensus_meta.cc:287] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.556s user 0.011s sys 0.000s
I20260619 21:16:15.344378 2422 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:15.384256 2422 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:15.385537 1735 raft_consensus.cc:1261] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260619 21:16:15.349622 2885 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:15.344771 2810 raft_consensus.cc:2726] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:15.345292 2809 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.378867 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.391613 1744 raft_consensus.cc:3037] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:15.346164 2693 raft_consensus.cc:2726] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:15.380656 2772 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:15.408897 2772 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.410272 2933 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.411092 2772 leader_election.cc:288] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.412560 1739 raft_consensus.cc:2396] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:15.412696 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.413838 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.414131 1739 raft_consensus.cc:1226] T 9980ccca53c3424db1585f12669d6d58 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260619 21:16:15.380555 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.374991 2900 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:15.385979 1735 pending_rounds.cc:76] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Aborting all ops after (but not including) 2
I20260619 21:16:15.344283 2602 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.346547 1661 raft_consensus.cc:1261] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:15.373706 1742 raft_consensus.cc:2396] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:15.423767 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.425288 1742 raft_consensus.cc:1226] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260619 21:16:15.426190 1635 leader_election.cc:302] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.293823 2859 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.393826 1782 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.352697 2956 consensus_queue.cc:1046] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260619 21:16:15.430915 1782 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.384783 2422 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:15.399353 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:15.401805 1741 raft_consensus.cc:1226] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:15.404675 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:15.371829 2805 raft_consensus.cc:2726] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.373186 2827 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.376577 2935 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:15.417362 1735 pending_rounds.cc:97] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260619 21:16:15.438340 1735 raft_consensus.cc:2866] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:15.419591 1739 raft_consensus.cc:1261] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260619 21:16:15.419775 1634 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.422156 1783 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.426602 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:15.370141 1662 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:15.416728 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.475876 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.428047 1635 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.431887 1736 raft_consensus.cc:1226] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:15.429183 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:15.433429 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:15.474381 1739 pending_rounds.cc:76] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: Aborting all ops after (but not including) 1
I20260619 21:16:15.477471 1739 pending_rounds.cc:97] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260619 21:16:15.477766 1739 raft_consensus.cc:2866] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:15.478667 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.479375 1666 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:15.480101 1818 raft_consensus.cc:3037] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:15.427466 2900 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260619 21:16:15.485477 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 57501.2us
W20260619 21:16:15.486099 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 66415.6us
I20260619 21:16:15.489454 1635 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.503587 1635 leader_election.cc:302] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.427626 1664 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:15.439992 2806 raft_consensus.cc:2781] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:15.473768 2935 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:15.503981 2893 raft_consensus.cc:2726] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.400745 2809 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.434100 2810 raft_consensus.cc:2726] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.510973 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.511663 2421 consensus_queue.cc:1035] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:15.516302 1782 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.517089 2886 raft_consensus.cc:2726] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.504189 2827 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260619 21:16:15.527865 2788 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.528131 1662 raft_consensus.cc:1261] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260619 21:16:15.528486 2788 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.530625 2788 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.535305 1818 raft_consensus.cc:2454] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:15.491657 2900 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:15.489449 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:15.545255 1635 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.489737 2805 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.449097 2939 consensus_queue.cc:1035] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260619 21:16:15.433972 2602 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:15.534267 2953 log.cc:924] Time spent T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.427s user 0.002s sys 0.002s
I20260619 21:16:15.556277 1860 tablet_bootstrap.cc:492] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:15.567562 1860 ts_tablet_manager.cc:1397] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.662s user 0.020s sys 0.022s
W20260619 21:16:15.557698 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 46813us
W20260619 21:16:15.547880 2930 consensus_meta.cc:287] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.666s user 0.007s sys 0.007s
I20260619 21:16:15.552908 2886 consensus_queue.cc:1035] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260619 21:16:15.570866 1635 leader_election.cc:302] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.545008 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.579926 1635 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.554562 2684 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:15.587842 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 71655.3us
W20260619 21:16:15.581976 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 159854us
I20260619 21:16:15.571370 1666 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:15.588587 1782 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.589675 1708 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.553707 2672 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.591794 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.592293 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.574100 2893 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.578672 2860 raft_consensus.cc:2781] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:15.608366 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:15.556806 1811 raft_consensus.cc:2454] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:15.611560 1744 raft_consensus.cc:2454] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:15.612198 1782 leader_election.cc:302] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.576047 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.614322 1634 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.591106 2865 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.593688 2856 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:15.620371 1813 raft_consensus.cc:1226] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:15.621229 1782 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.591116 2810 raft_consensus.cc:2726] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:15.590956 2421 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:15.593207 2726 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:15.593510 2713 raft_consensus.cc:2726] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:15.627925 1635 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:15.719154 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 110685us
I20260619 21:16:15.614292 2860 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:15.674021 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 59783.2us
I20260619 21:16:15.577276 2754 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.618577 2795 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:15.577147 2923 raft_consensus.cc:2726] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:15.720744 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 612) took 1068 ms. Trace:
W20260619 21:16:15.720878 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 93074.4us
I20260619 21:16:15.721611 1815 raft_consensus.cc:1261] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260619 21:16:15.623247 1664 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 6.
I20260619 21:16:15.623839 2931 raft_consensus.cc:3037] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:15.597420 2930 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.746688 1635 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.748438 1812 rpcz_store.cc:276] 0619 21:16:14.652556 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:14.653617 (+ 1061us) service_pool.cc:227] Handling call
0619 21:16:14.653932 (+ 315us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:14.905707 (+251775us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:15.720462 (+814755us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400115dd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:15.720680 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":562352384}
W20260619 21:16:15.732528 2963 log.cc:924] Time spent T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.305s user 0.001s sys 0.000s
I20260619 21:16:15.752969 1860 raft_consensus.cc:348] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.753775 1860 raft_consensus.cc:374] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:15.754038 1860 raft_consensus.cc:729] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:15.754681 1860 consensus_queue.cc:260] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.756311 1860 ts_tablet_manager.cc:1428] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.188s user 0.005s sys 0.000s
I20260619 21:16:15.757447 1708 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.712708 2805 raft_consensus.cc:2781] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 21:16:15.721788 2859 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.737215 2860 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:15.733467 2810 raft_consensus.cc:2726] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.761217 1743 raft_consensus.cc:2354] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:15.764269 2754 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.765998 2930 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.768226 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.697719 2865 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.750154 2903 raft_consensus.cc:2781] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:15.737578 2726 raft_consensus.cc:686] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:15.771796 1860 tablet_bootstrap.cc:492] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:15.779222 2602 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.760586 2933 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260619 21:16:15.780088 2933 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:15.780485 2933 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:15.786840 2672 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.787596 2809 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:15.789613 2784 consensus_meta.cc:287] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.507s user 0.011s sys 0.000s
I20260619 21:16:15.790129 2784 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.791744 2861 raft_consensus.cc:2726] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.795778 2726 consensus_queue.cc:237] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.802532 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:15.738212 2942 raft_consensus.cc:3032] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:15.751892 2684 raft_consensus.cc:2781] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:15.792668 2784 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.809790 1667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 566) took 1288 ms. Trace:
I20260619 21:16:15.778383 2903 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:15.820060 1667 rpcz_store.cc:276] 0619 21:16:14.521713 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:14.522759 (+ 1046us) service_pool.cc:227] Handling call
0619 21:16:15.802333 (+1279574us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a001b0 00007f71a39ffa94 00007f719d0e7c04 00007f719d0e7b92
0619 21:16:15.809728 (+ 7395us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":36607872}
I20260619 21:16:15.821815 1708 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:15.823035 2861 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:15.823956 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.825440 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:15.825877 1667 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:15.806772 2942 raft_consensus.cc:729] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:15.804374 2684 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:15.777060 2923 consensus_queue.cc:1046] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260619 21:16:15.827414 2903 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:15.877744 1667 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:15.882977 1708 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.854211 2942 consensus_queue.cc:260] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.902901 1667 raft_consensus.cc:1226] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: []
W20260619 21:16:15.903575 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 282395us
I20260619 21:16:15.904556 1782 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.862695 2684 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:15.908826 1665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 563) took 1418 ms (client timeout 1754 ms). Trace:
I20260619 21:16:15.910071 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:15.909360 1665 rpcz_store.cc:269] 0619 21:16:14.490832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:14.492359 (+ 1527us) service_pool.cc:227] Handling call
0619 21:16:15.908601 (+1416242us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b54000a9e54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:15.908753 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":29058432}
I20260619 21:16:15.915035 1708 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:15.916229 1665 raft_consensus.cc:1226] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:15.919052 1782 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.920359 2923 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:15.922113 2865 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.922559 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.923189 1741 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:15.926602 1665 raft_consensus.cc:1261] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:15.927235 2772 raft_consensus.cc:2726] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:15.928280 2883 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:15.929188 1782 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.930073 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:15.931154 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.932126 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.932767 1782 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.933012 1741 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:15.933555 2979 raft_consensus.cc:2726] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:15.934247 1741 raft_consensus.cc:1261] T c8a530b7f9264d519f65caf431a0a478 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:15.934952 2923 raft_consensus.cc:2726] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:15.935398 1741 raft_consensus.cc:1226] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: []
@ 0x7f719bd85e90 testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
I20260619 21:16:15.936136 2979 consensus_queue.cc:1046] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:15.937206 2602 raft_consensus.cc:3032] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:15.937577 2602 raft_consensus.cc:729] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:15.938509 2602 consensus_queue.cc:260] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.939544 2942 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:15.940425 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:15.942946 1782 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.943747 2979 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:15.945499 1782 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.946623 1782 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.947820 2979 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:15.949981 1782 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.882577 2859 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:15.932778 1736 raft_consensus.cc:2379] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 1: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:15.916983 2784 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:15.906324 2929 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:15.973950 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.975369 1783 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.981012 1634 leader_election.cc:302] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.981609 1782 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.932135 2772 raft_consensus.cc:2726] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:15.971408 2923 raft_consensus.cc:2781] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:15.972180 2784 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:15.987987 1662 raft_consensus.cc:1261] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260619 21:16:15.987491 1782 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.973659 2805 raft_consensus.cc:686] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:15.991681 2683 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:15.992069 2784 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:15.991781 2805 consensus_queue.cc:237] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.992251 2683 raft_consensus.cc:2726] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:15.998294 1782 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:15.997689 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:15.999518 1736 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:16.003618 1634 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:15.978034 2856 raft_consensus.cc:686] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:16.012389 1736 raft_consensus.cc:1226] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260619 21:16:15.982769 2929 raft_consensus.cc:686] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:15.970719 2859 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:15.986754 2923 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:16.021806 2981 raft_consensus.cc:2726] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:16.023451 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.024173 2981 raft_consensus.cc:2781] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:16.024700 2981 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:16.025230 2981 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.024966 2929 consensus_queue.cc:237] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.013198 2980 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:16.029072 2980 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:16.029582 2980 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.029675 2859 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:15.985983 2925 consensus_queue.cc:1046] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260619 21:16:16.035069 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 59796.1us
I20260619 21:16:16.019372 2923 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:16.018966 2876 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.019205 2772 raft_consensus.cc:2781] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:15.995054 2935 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.034171 2759 raft_consensus.cc:2781] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:16.024299 1811 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:16.018081 2982 raft_consensus.cc:2726] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:16.051213 2925 raft_consensus.cc:3032] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:16.093822 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 606) took 1123 ms. Trace:
I20260619 21:16:16.094357 1740 rpcz_store.cc:276] 0619 21:16:14.970875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:14.972138 (+ 1263us) service_pool.cc:227] Handling call
0619 21:16:14.973258 (+ 1120us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:15.093753 (+120495us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.093784 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:15.093822 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:15.093849 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.093570 (+999721us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54002d5ad4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:16.093618 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.093756 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":276580224}
I20260619 21:16:16.072649 2984 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.077880 2772 raft_consensus.cc:686] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:16.034577 2885 consensus_queue.cc:1046] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260619 21:16:16.090874 2925 raft_consensus.cc:729] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:16.117622 2925 consensus_queue.cc:260] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.118891 2925 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:16.118440 2933 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.121523 2903 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.123701 2903 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:16.125274 2983 log.cc:924] Time spent T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Append to log took a long time: real 0.112s user 0.001s sys 0.002s
I20260619 21:16:16.071874 2876 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.131887 2860 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.132979 2876 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.104352 2759 raft_consensus.cc:686] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:16.134063 2860 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:16.134210 2759 consensus_queue.cc:237] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.135924 2754 raft_consensus.cc:3032] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:16.136428 2754 raft_consensus.cc:729] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:16.110972 2935 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.081501 2987 consensus_queue.cc:1035] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260619 21:16:16.114146 2981 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.115356 2784 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.138654 2806 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:16.139097 2806 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.139837 2981 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:16.139945 2784 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.141168 2931 raft_consensus.cc:2726] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:16.076103 2856 consensus_queue.cc:237] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.147648 2933 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.143381 2772 consensus_queue.cc:237] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.149767 2922 raft_consensus.cc:3032] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:16.150224 2922 raft_consensus.cc:729] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:16.151204 2922 consensus_queue.cc:260] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.151973 2980 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.154101 2980 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.155073 2923 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.157248 2923 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:16.106216 2984 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.158607 2754 consensus_queue.cc:260] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.160082 2984 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.160218 2754 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.144536 2734 raft_consensus.cc:3032] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:16.161237 2734 raft_consensus.cc:729] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:16.147341 2979 raft_consensus.cc:2726] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:16.162304 2734 consensus_queue.cc:260] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:16.156412 2900 consensus_meta.cc:287] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.600s user 0.004s sys 0.006s
I20260619 21:16:16.163877 2734 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.164165 2900 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:16.160301 2957 log.cc:924] Time spent T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.731s user 0.003s sys 0.009s
I20260619 21:16:16.166406 2900 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.152791 2922 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:16.240979 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 539) took 1102 ms. Trace:
I20260619 21:16:16.242096 1661 rpcz_store.cc:276] 0619 21:16:15.138199 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.623373 (+485174us) service_pool.cc:227] Handling call
0619 21:16:15.741785 (+118412us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:15.761430 (+ 19645us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.766351 (+ 4921us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:15.777263 (+ 10912us) log.cc:842] Serialized 26 byte log entry
0619 21:16:15.969283 (+192020us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:16.001398 (+ 32115us) log.cc:842] Serialized 14 byte log entry
0619 21:16:16.015066 (+ 13668us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.018994 (+ 3928us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.240066 (+221072us) raft_consensus.cc:1671] finished
0619 21:16:16.240144 (+ 78us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.240550 (+ 406us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":11884,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":3016,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":1844,"thread_start_us":137717,"threads_started":1,"wal-append.queue_time_us":222789}
@ 0x7f719bd63a92 testing::Test::Run() at ??:0
I20260619 21:16:16.245213 1664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 541) took 1091 ms. Trace:
I20260619 21:16:16.245868 1664 rpcz_store.cc:276] 0619 21:16:15.153245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.753312 (+600067us) service_pool.cc:227] Handling call
0619 21:16:15.753711 (+ 399us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:15.811343 (+ 57632us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.811378 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:15.811961 (+ 583us) log.cc:842] Serialized 26 byte log entry
0619 21:16:15.973578 (+161617us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:15.976589 (+ 3011us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.244181 (+267592us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b5400055fd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:16.244261 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.244381 (+ 120us) raft_consensus.cc:1671] finished
0619 21:16:16.244402 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.244931 (+ 529us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1208,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":157,"spinlock_wait_cycles":127949696,"thread_start_us":144574,"threads_started":1,"wal-append.queue_time_us":218176}
I20260619 21:16:16.248760 1663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 514) took 1315 ms. Trace:
I20260619 21:16:16.249248 1663 rpcz_store.cc:276] 0619 21:16:14.933662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:14.958212 (+ 24550us) service_pool.cc:227] Handling call
0619 21:16:16.248286 (+1290074us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54001dbfd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:16.248514 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":196708224}
I20260619 21:16:16.248550 1661 raft_consensus.cc:1261] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:16.252456 1665 raft_consensus.cc:1226] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 5. Current term is 6. Ops: []
I20260619 21:16:16.255335 1665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 557) took 1006 ms. Trace:
I20260619 21:16:16.255702 1665 rpcz_store.cc:276] 0619 21:16:15.249193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.929907 (+680714us) service_pool.cc:227] Handling call
0619 21:16:15.930154 (+ 247us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:16.253040 (+322886us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.255030 (+ 1990us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54000a9e54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:16.255274 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":739630464}
I20260619 21:16:16.260048 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 578) took 1333 ms. Trace:
I20260619 21:16:16.260519 1669 rpcz_store.cc:276] 0619 21:16:14.926867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.077712 (+150845us) service_pool.cc:227] Handling call
0619 21:16:15.078170 (+ 458us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:15.092436 (+ 14266us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.092462 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:15.099643 (+ 7181us) log.cc:842] Serialized 26 byte log entry
0619 21:16:15.185038 (+ 85395us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:15.185095 (+ 57us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:15.185183 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.159695 (+974512us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719e740bee 00007f719e73ff36 00007f719e73fb58 00007f719e749551 00007f719e749509 00007f719e749499
0619 21:16:16.259461 (+ 99766us) raft_consensus.cc:1671] finished
0619 21:16:16.259498 (+ 37us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.259844 (+ 346us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18965,"raft.queue_time_us":13906,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":9350,"spinlock_wait_cycles":10402560,"thread_start_us":53268,"threads_started":1,"wal-append.queue_time_us":258576}
I20260619 21:16:16.263684 1669 raft_consensus.cc:1226] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:16.265909 1660 raft_consensus.cc:1226] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:16.267656 1660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 524) took 1326 ms. Trace:
I20260619 21:16:16.267997 1660 rpcz_store.cc:276] 0619 21:16:14.941426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.244651 (+303225us) service_pool.cc:227] Handling call
0619 21:16:16.265287 (+1020636us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:16.266416 (+ 1129us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.267398 (+ 982us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400106dd0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabda09 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:16.267584 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2346052480}
I20260619 21:16:16.273082 1735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 612) took 1293 ms. Trace:
I20260619 21:16:16.097586 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.275308 1737 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:16.278342 1740 raft_consensus.cc:1226] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 5. Current term is 6. Ops: [5.2-5.2]
W20260619 21:16:16.279186 1710 consensus_peers.cc:487] T d80394dabe604f8898e7b24149fc624f P d53146739e3240239c0a3d5ea90e07fe -> Peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Couldn't send request to peer b2e7f02778ec43bdbca03dd013687c76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260619 21:16:16.285094 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.34208e+06us
I20260619 21:16:16.288551 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.289741 1664 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 6: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:16.290608 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.291590 1661 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:16.281378 1741 raft_consensus.cc:1261] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260619 21:16:16.282037 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.282557 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 631) took 1017 ms. Trace:
I20260619 21:16:16.273695 1735 rpcz_store.cc:276] 0619 21:16:14.979402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.064920 (+ 85518us) service_pool.cc:227] Handling call
0619 21:16:15.098769 (+ 33849us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:15.473361 (+374592us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.272231 (+798870us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002b7d54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:16.272716 (+ 485us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":658289024}
W20260619 21:16:16.097992 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 108526us
I20260619 21:16:16.296553 1660 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Advancing to term 7
W20260619 21:16:16.302788 1737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 543) took 1202 ms (client timeout 1553 ms). Trace:
W20260619 21:16:16.303488 1737 rpcz_store.cc:269] 0619 21:16:15.100511 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.137609 (+ 37098us) service_pool.cc:227] Handling call
0619 21:16:16.302526 (+1164917us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540032df54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:16.302682 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2453677312}
I20260619 21:16:16.294440 1744 rpcz_store.cc:276] 0619 21:16:15.265225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.809527 (+544302us) service_pool.cc:227] Handling call
0619 21:16:15.856682 (+ 47155us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:15.858416 (+ 1734us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.858463 (+ 47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:15.859069 (+ 606us) log.cc:842] Serialized 26 byte log entry
0619 21:16:16.017526 (+158457us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:16.017576 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.017669 (+ 93us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.281985 (+264316us) raft_consensus.cc:1671] finished
0619 21:16:16.282024 (+ 39us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.282310 (+ 286us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":940,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4091,"thread_start_us":130966,"threads_started":1,"wal-append.queue_time_us":213325}
I20260619 21:16:16.315639 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 545) took 1214 ms. Trace:
I20260619 21:16:16.315937 1736 raft_consensus.cc:1226] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:16.316201 1743 rpcz_store.cc:276] 0619 21:16:15.100837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.176261 (+ 75424us) service_pool.cc:227] Handling call
0619 21:16:16.315063 (+1138802us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54002b8754. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:16.315377 (+ 314us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1317086976}
I20260619 21:16:16.359550 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 585) took 1253 ms. Trace:
I20260619 21:16:16.359694 1782 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.359898 1663 raft_consensus.cc:1261] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260619 21:16:16.361370 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 57325.4us
I20260619 21:16:16.361827 2809 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:16.362926 2809 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.364137 1782 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.365494 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.365777 2979 raft_consensus.cc:2781] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:16.366384 2979 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:16.366813 2979 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.368324 2856 consensus_queue.cc:1035] T 246c3d015ec242eca8bfaf5a076692b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:16.369603 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.370546 1817 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.370340 2994 consensus_queue.cc:1035] T 9980ccca53c3424db1585f12669d6d58 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:16.360196 1666 rpcz_store.cc:276] 0619 21:16:15.097127 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.595936 (+498809us) service_pool.cc:227] Handling call
0619 21:16:15.596319 (+ 383us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:15.601459 (+ 5140us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.601879 (+ 420us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:15.611409 (+ 9530us) log.cc:842] Serialized 26 byte log entry
0619 21:16:16.253650 (+642241us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b1800079600. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e745411 00007f719ea925cf
0619 21:16:16.270762 (+ 17112us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:16.270854 (+ 92us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.271006 (+ 152us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.316786 (+ 45780us) raft_consensus.cc:1671] finished
0619 21:16:16.316815 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.350538 (+ 33723us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":24040,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":15813,"spinlock_wait_cycles":21537024,"thread_start_us":16664,"threads_started":1,"wal-append.queue_time_us":652285}
I20260619 21:16:16.379377 2719 consensus_queue.cc:1035] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:16.380613 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.382767 2883 consensus_queue.cc:1046] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:16:16.188542 2806 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.364148 1812 raft_consensus.cc:2454] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:16.382774 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:16:16.390344 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 83321.7us
I20260619 21:16:16.390511 2805 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.391408 2930 raft_consensus.cc:3032] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:16.391281 2805 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.391913 2930 raft_consensus.cc:729] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:16.393565 2806 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.393787 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.393362 2930 consensus_queue.cc:260] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.394833 1810 raft_consensus.cc:2379] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 4: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:16.384604 2883 consensus_queue.cc:1046] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260619 21:16:16.395699 2930 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.401958 2861 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:16.402573 2805 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.404516 2861 raft_consensus.cc:3032] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:16.409874 1815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 581) took 1507 ms. Trace:
I20260619 21:16:16.410187 2902 raft_consensus.cc:2726] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:16.410351 1815 rpcz_store.cc:276] 0619 21:16:14.902652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:14.902771 (+ 119us) service_pool.cc:227] Handling call
0619 21:16:15.721233 (+818462us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:15.750474 (+ 29241us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.409414 (+658940us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b5400115dd0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabda09 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:16.409650 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1881285376}
I20260619 21:16:16.417336 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.422062 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.423372 1743 raft_consensus.cc:2379] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
W20260619 21:16:16.428848 2969 log.cc:924] Time spent T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260619 21:16:16.430357 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.436506 2602 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.441931 1668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 528) took 1497 ms. Trace:
I20260619 21:16:16.412130 2861 raft_consensus.cc:729] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
W20260619 21:16:16.446899 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 89876.2us
W20260619 21:16:16.450687 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 86781us
I20260619 21:16:16.452896 2759 consensus_queue.cc:1035] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:16.453773 1635 leader_election.cc:302] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.453881 2759 consensus_queue.cc:1225] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer d53146739e3240239c0a3d5ea90e07fe log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260619 21:16:16.453923 2810 consensus_queue.cc:1035] T f756a9c8b10a4992950255915336bb2c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:16.456092 2902 raft_consensus.cc:2781] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:16.457605 2902 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:16.458062 2902 raft_consensus.cc:3037] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:16.451941 1860 tablet_bootstrap.cc:654] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:16.460742 2809 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.452812 1782 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.462695 2759 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260619 21:16:16.464322 1782 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.456707 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 628) took 1248 ms. Trace:
I20260619 21:16:16.467269 1662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 589) took 1073 ms. Trace:
I20260619 21:16:16.442409 1668 rpcz_store.cc:276] 0619 21:16:14.944268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.259806 (+315538us) service_pool.cc:227] Handling call
0619 21:16:15.260110 (+ 304us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:15.289347 (+ 29237us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:15.289394 (+ 47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:15.289970 (+ 576us) log.cc:842] Serialized 26 byte log entry
0619 21:16:16.269364 (+979394us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b18002cc740. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e745411 00007f719ea925cf
0619 21:16:16.372923 (+103559us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:16.409488 (+ 36565us) log.cc:842] Serialized 14 byte log entry
0619 21:16:16.409758 (+ 270us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.409864 (+ 106us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.440639 (+ 30775us) raft_consensus.cc:1671] finished
0619 21:16:16.441515 (+ 876us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.441718 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1903,"raft.queue_time_us":31218,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":49126,"spinlock_wait_cycles":56695680,"thread_start_us":47818,"threads_started":1,"wal-append.queue_time_us":1126254}
I20260619 21:16:16.469831 1662 rpcz_store.cc:276] 0619 21:16:15.340356 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:16.243320 (+902964us) service_pool.cc:227] Handling call
0619 21:16:16.244196 (+ 876us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:16.244929 (+ 733us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:16.244984 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:16.246187 (+ 1203us) log.cc:842] Serialized 26 byte log entry
0619 21:16:16.354210 (+108023us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:16.354284 (+ 74us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.354587 (+ 303us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.400599 (+ 46012us) raft_consensus.cc:1671] finished
0619 21:16:16.400620 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.413926 (+ 13306us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":120355,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":365,"thread_start_us":100576,"threads_started":1,"wal-append.queue_time_us":109981}
I20260619 21:16:16.468230 1738 rpcz_store.cc:276] 0619 21:16:15.208397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.614730 (+406333us) service_pool.cc:227] Handling call
0619 21:16:15.615049 (+ 319us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:15.615256 (+ 207us) raft_consensus.cc:1520] Early marking committed up to index 1
0619 21:16:16.310770 (+695514us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b18000fd900. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719e74d82c 00007f719bb789dc 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb93155 00007f71a2493167 00007f719eac8807 00007f719ead7f0f
0619 21:16:16.311029 (+ 259us) log.cc:842] Serialized 14 byte log entry
0619 21:16:16.449210 (+138181us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:16.456088 (+ 6878us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:16.456174 (+ 86us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.456293 (+ 119us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.456455 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":62203,"prepare.queue_time_us":103148,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":96111,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":133,"spinlock_wait_cycles":7170944,"thread_start_us":142709,"threads_started":2}
I20260619 21:16:16.483769 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.484565 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.485378 1815 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:16.473667 2759 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
W20260619 21:16:16.490125 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 97158.9us
I20260619 21:16:16.491441 1635 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:16.495525 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.493234 2809 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:16.504269 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.507344 2977 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:16.510847 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
W20260619 21:16:16.513613 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 50725.5us
I20260619 21:16:16.513831 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.510713 1782 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.516058 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.519367 2421 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:16.520246 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 589) took 1122 ms. Trace:
I20260619 21:16:16.520069 2861 consensus_queue.cc:260] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.520686 1813 rpcz_store.cc:276] 0619 21:16:15.397347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.407235 (+ 9888us) service_pool.cc:227] Handling call
0619 21:16:15.407571 (+ 336us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:15.630953 (+223382us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.493274 (+862321us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400260f54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:16.520045 (+ 26771us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":488837888}
I20260619 21:16:16.524561 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.525338 1816 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:16.533608 1741 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 6: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:16.532985 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 553) took 1399 ms. Trace:
I20260619 21:16:16.536485 1744 raft_consensus.cc:3037] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.534128 1817 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 6.
I20260619 21:16:16.539171 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.540868 1811 raft_consensus.cc:1261] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:16.543293 2754 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:16.543784 2754 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.544201 2754 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:16.520176 2421 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.511081 2809 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.549425 2809 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:16.549718 2902 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.551918 2902 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.553464 2672 consensus_queue.cc:1035] T da1270002c3c45988c3ac4fbeafaf019 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:16:16.534912 1742 rpcz_store.cc:276] 0619 21:16:15.112133 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.425880 (+313747us) service_pool.cc:227] Handling call
0619 21:16:15.426135 (+ 255us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:15.426335 (+ 200us) raft_consensus.cc:1520] Early marking committed up to index 1
0619 21:16:15.459578 (+ 33243us) log.cc:842] Serialized 14 byte log entry
0619 21:16:16.293266 (+833688us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b18002bc120. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e74557b 00007f719eac888a
0619 21:16:16.478340 (+185074us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:16.482598 (+ 4258us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:16.490211 (+ 7613us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.500332 (+ 10121us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.510808 (+ 10476us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":72460,"prepare.queue_time_us":13087,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":61964,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":3052,"spinlock_wait_cycles":17917824,"thread_start_us":57852,"threads_started":1,"wal-append.queue_time_us":941713}
I20260619 21:16:16.544010 1782 leader_election.cc:302] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:16.576277 3012 log.cc:924] Time spent T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.107s user 0.004s sys 0.000s
I20260619 21:16:16.557237 2772 consensus_queue.cc:1035] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:16.582821 1660 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 7.
I20260619 21:16:16.591396 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.592473 1811 raft_consensus.cc:3037] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:16.597093 1738 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
W20260619 21:16:16.597345 2684 consensus_meta.cc:287] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.642s user 0.008s sys 0.011s
I20260619 21:16:16.597815 2684 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.600178 2684 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.596531 2861 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.601116 1667 raft_consensus.cc:1226] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:16.603541 1667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 548) took 1448 ms. Trace:
I20260619 21:16:16.603878 1667 rpcz_store.cc:276] 0619 21:16:15.155228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.903984 (+748756us) service_pool.cc:227] Handling call
0619 21:16:15.925624 (+ 21640us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:16.601643 (+676019us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.603255 (+ 1612us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b540036e554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:16.603471 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1552344448}
I20260619 21:16:16.605700 1635 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:16.548734 2421 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.516647 1661 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:16.608736 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.615008 2683 raft_consensus.cc:2781] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260619 21:16:16.615732 2683 raft_consensus.cc:686] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [term 6 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
W20260619 21:16:16.615899 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 72033.9us
I20260619 21:16:16.617022 2683 consensus_queue.cc:237] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.617669 2856 consensus_queue.cc:1046] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:16.619876 2975 consensus_queue.cc:1035] T 32ea84a8ad12422f8ec0ede64598fa44 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:16.622013 2975 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:16.622004 2421 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:16.622443 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.623000 2421 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.623656 2975 consensus_queue.cc:1046] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260619 21:16:16.625687 1634 leader_election.cc:334] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.553s (SENT)
I20260619 21:16:16.626176 1634 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.612987 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 600) took 1002 ms. Trace:
I20260619 21:16:16.627992 1669 rpcz_store.cc:276] 0619 21:16:15.610702 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:16.264641 (+653939us) service_pool.cc:227] Handling call
0619 21:16:16.312224 (+ 47583us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:16.409216 (+ 96992us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:16.409254 (+ 38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:16.409791 (+ 537us) log.cc:842] Serialized 26 byte log entry
0619 21:16:16.447413 (+ 37622us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:16.447452 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:16.447530 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:16.612752 (+165222us) raft_consensus.cc:1671] finished
0619 21:16:16.612781 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:16.612908 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2176,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":30003,"thread_start_us":37135,"threads_started":1,"wal-append.queue_time_us":106500}
I20260619 21:16:16.631603 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.627699 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
@ 0x7f719bd64da9 testing::TestInfo::Run() at ??:0
I20260619 21:16:16.634601 2684 raft_consensus.cc:2726] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:16.655198 1813 raft_consensus.cc:1226] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:16.654480 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.658942 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.659835 1816 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:16.661682 1818 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:16.662742 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.660840 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.662674 1812 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:16.664888 1816 raft_consensus.cc:2379] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 6: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:16.669028 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.671880 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.686957 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.687906 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.689111 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.693349 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:16.675447 2421 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.701359 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:16.703140 1744 raft_consensus.cc:2454] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 6.
I20260619 21:16:16.641695 2734 consensus_queue.cc:1046] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260619 21:16:16.714058 1635 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:16.673561 2930 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:16.716687 1743 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 7 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260619 21:16:16.682679 2975 consensus_queue.cc:1035] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260619 21:16:16.726212 2930 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.727278 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.734416 1860 tablet_bootstrap.cc:492] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:16.709646 2684 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:16.738456 2422 consensus_meta.cc:287] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.150s user 0.014s sys 0.007s
I20260619 21:16:16.738790 2726 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:16.758327 1809 raft_consensus.cc:2454] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:16.679842 2979 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.758684 1812 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
W20260619 21:16:16.763602 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 45962.9us
I20260619 21:16:16.764676 1782 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.774561 1860 ts_tablet_manager.cc:1397] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 1.013s user 0.020s sys 0.048s
I20260619 21:16:16.776214 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.771070 1811 raft_consensus.cc:2454] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:16.781032 1634 leader_election.cc:398] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Vote denied by peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475) with higher term. Message: Invalid argument: T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3.
I20260619 21:16:16.707476 2754 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.786180 1813 raft_consensus.cc:1226] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:16:16.794032 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.794672 1811 raft_consensus.cc:3037] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:16.798609 1816 raft_consensus.cc:1261] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:16.792982 1661 raft_consensus.cc:1261] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260619 21:16:16.797925 3014 log.cc:924] Time spent T 84119be03678450983c7d5a686bd3c86 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.098s user 0.000s sys 0.003s
I20260619 21:16:16.807313 2903 consensus_queue.cc:1046] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260619 21:16:16.782873 2726 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.773804 2422 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.782091 2684 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.815017 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.817070 2713 raft_consensus.cc:2741] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260619 21:16:16.817719 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 48159.5us
I20260619 21:16:16.841923 2684 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.842581 1634 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.842515 2684 consensus_queue.cc:1046] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:16.805438 2925 raft_consensus.cc:2726] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:16.844063 1739 raft_consensus.cc:1226] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 5. Current term is 6. Ops: []
W20260619 21:16:16.844226 3017 log.cc:924] Time spent T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.072s user 0.001s sys 0.000s
I20260619 21:16:16.805603 2797 raft_consensus.cc:3032] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:16.805989 2984 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:16.827646 1811 raft_consensus.cc:2454] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:16.834112 1816 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:16.838681 1635 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:16.842967 2979 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:16.845295 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.847379 2797 raft_consensus.cc:729] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:16.852269 1740 raft_consensus.cc:2379] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:16.857975 2865 raft_consensus.cc:2726] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:16.859747 1635 leader_election.cc:302] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:16.861308 2902 raft_consensus.cc:2781] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:16.861837 2902 raft_consensus.cc:686] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:16.861869 2797 consensus_queue.cc:260] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.862941 2902 consensus_queue.cc:237] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.863760 2797 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.865576 1634 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.866449 1708 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:16.866882 2810 raft_consensus.cc:2781] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:16.867398 1634 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:16.870182 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.844862 2422 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.872910 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:16.844079 2930 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.874274 2865 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260619 21:16:16.874732 2759 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:16.875368 2775 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 7 FOLLOWER]: Leader election won for term 7
I20260619 21:16:16.882771 2929 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:16.883163 2942 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:16.889986 1812 raft_consensus.cc:1226] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260619 21:16:16.891808 1860 raft_consensus.cc:348] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.893623 2726 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.901674 1816 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:16.907186 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:16.908047 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:16.908419 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.909444 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.910322 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:16.910889 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.911427 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.914816 1818 raft_consensus.cc:1261] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:16.912413 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.916625 1708 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:16.923785 1666 raft_consensus.cc:3037] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:16.916857 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:16:16.917081 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 49757.2us
I20260619 21:16:16.917744 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:16.918130 1710 leader_election.cc:341] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260619 21:16:16.925534 1634 leader_election.cc:302] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:16.921043 1860 raft_consensus.cc:374] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:16.928340 1860 raft_consensus.cc:729] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:16.925134 1710 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:16.931547 1814 raft_consensus.cc:1261] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:16.931885 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:16.932312 2806 raft_consensus.cc:3032] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:16.932821 2806 raft_consensus.cc:729] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:16.932420 2929 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.934768 2929 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:16.935117 1860 consensus_queue.cc:260] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.936137 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:16.928314 2775 raft_consensus.cc:686] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [term 7 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:16.941484 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 1764e19eb06140f8bbdb1c3aab252d9a
I20260619 21:16:16.942026 2775 consensus_queue.cc:237] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.943538 2754 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:16.943795 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 5ae5f9625019495d85bc13c82d7f621b
I20260619 21:16:16.945299 1657 tablet_service.cc:1514] Processing DeleteTablet for tablet 1764e19eb06140f8bbdb1c3aab252d9a with delete_type TABLET_DATA_DELETED (Replaced by 21131e98f1f94e8cac1998e6c32aa089 at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:16.945879 2984 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.946336 2984 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:16.947263 2809 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:16.914592 2683 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260619 21:16:16.951092 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 83440.8us
I20260619 21:16:16.973575 1815 raft_consensus.cc:1261] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:16.949187 1665 raft_consensus.cc:1226] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
W20260619 21:16:16.965929 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 47850.3us
I20260619 21:16:16.992447 1666 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:16.948493 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae5f9625019495d85bc13c82d7f621b with delete_type TABLET_DATA_DELETED (Replaced by 6ae495cfaaca419c8611fe233762a02a at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.002234 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet e3b2adcb38244b60bdd7ae5d5642f641
I20260619 21:16:16.955094 1664 raft_consensus.cc:2354] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5.
I20260619 21:16:17.009836 2860 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.009958 1710 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74
I20260619 21:16:17.010404 2860 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.010921 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 0dea6228e49c44b9889aa709158ef557
I20260619 21:16:17.010728 2929 consensus_queue.cc:1035] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260619 21:16:17.012244 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 9ae81d060b714ae98341d6b5e2669f3f
I20260619 21:16:17.012521 2860 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:17.013435 3029 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.013973 3029 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.015877 1669 raft_consensus.cc:1261] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 21:16:16.951431 2726 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:16.952113 2930 raft_consensus.cc:2726] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:16.967983 2683 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:16.972926 2942 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.020498 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 8f709f3b1e904d1ba8bf5592d77cc8d6
I20260619 21:16:16.950634 1808 tablet_service.cc:1514] Processing DeleteTablet for tablet 1764e19eb06140f8bbdb1c3aab252d9a with delete_type TABLET_DATA_DELETED (Replaced by 21131e98f1f94e8cac1998e6c32aa089 at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.004873 2421 raft_consensus.cc:2726] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260619 21:16:17.033901 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 47593us
W20260619 21:16:17.033946 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 251737us
I20260619 21:16:17.027545 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:17.045477 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b2adcb38244b60bdd7ae5d5642f641 with delete_type TABLET_DATA_DELETED (Replaced by 18756ff2e1e84eb5b7139ecabf689d6a at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.046627 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae5f9625019495d85bc13c82d7f621b with delete_type TABLET_DATA_DELETED (Replaced by 6ae495cfaaca419c8611fe233762a02a at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.025334 2683 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:17.049710 1664 raft_consensus.cc:1261] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:17.041882 2772 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.050541 2772 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.055991 1669 raft_consensus.cc:1226] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260619 21:16:17.057740 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 246c3d015ec242eca8bfaf5a076692b1
I20260619 21:16:17.028249 2929 raft_consensus.cc:2741] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260619 21:16:17.027357 2726 raft_consensus.cc:686] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:17.071036 1808 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dea6228e49c44b9889aa709158ef557 with delete_type TABLET_DATA_DELETED (Replaced by 84b6bbbb60204347824250a8d71e6d3c at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.071110 1801 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b2adcb38244b60bdd7ae5d5642f641 with delete_type TABLET_DATA_DELETED (Replaced by 18756ff2e1e84eb5b7139ecabf689d6a at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.071290 1654 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dea6228e49c44b9889aa709158ef557 with delete_type TABLET_DATA_DELETED (Replaced by 84b6bbbb60204347824250a8d71e6d3c at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.072345 1806 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae81d060b714ae98341d6b5e2669f3f with delete_type TABLET_DATA_DELETED (Replaced by 331f6581b0ac4c3bbab86ecc34ececfb at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.052404 2865 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:17.074925 2942 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.043570 2809 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.074496 2865 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:17.076030 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 66190.1us
I20260619 21:16:17.021533 2810 raft_consensus.cc:686] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:17.093746 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet cfa69d158f2b471e8638824d3d9c757b
I20260619 21:16:17.093680 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:17.099221 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.099670 1662 raft_consensus.cc:1261] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260619 21:16:17.067725 2684 raft_consensus.cc:482] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:17.093546 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae81d060b714ae98341d6b5e2669f3f with delete_type TABLET_DATA_DELETED (Replaced by 331f6581b0ac4c3bbab86ecc34ececfb at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.109088 1667 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:17.109582 2726 consensus_queue.cc:237] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.110504 2865 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.111093 1806 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 with delete_type TABLET_DATA_DELETED (Replaced by a7a69751ebea42d5950577710f9b1301 at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.075541 2860 consensus_queue.cc:1046] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260619 21:16:17.103549 1735 raft_consensus.cc:1261] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:17.113365 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.064702 2930 consensus_queue.cc:1035] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260619 21:16:17.089427 2902 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.112712 2806 consensus_queue.cc:260] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.118192 1860 ts_tablet_manager.cc:1428] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.343s user 0.009s sys 0.001s
I20260619 21:16:17.119050 2772 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:17.122313 3029 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:17.123193 1738 raft_consensus.cc:1261] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260619 21:16:17.125330 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 48665.6us
I20260619 21:16:17.113219 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:17.125771 2684 raft_consensus.cc:504] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.126227 1739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 634) took 1818 ms. Trace:
I20260619 21:16:17.126502 2810 consensus_queue.cc:237] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.127166 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.136037 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet b249d6b40f6943e3bb7235b7df99e1b1
I20260619 21:16:17.156432 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.157346 1735 raft_consensus.cc:2379] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 5: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:17.148138 1739 rpcz_store.cc:276] 0619 21:16:15.307669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:15.816378 (+508709us) service_pool.cc:227] Handling call
0619 21:16:15.816634 (+ 256us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:16.844493 (+1027859us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:17.125902 (+281409us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400030054. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:17.126168 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2361755520}
I20260619 21:16:17.158802 2893 consensus_queue.cc:1035] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:17.160642 3042 tablet_replica.cc:330] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: stopping tablet replica
I20260619 21:16:17.161329 1782 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.162262 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:17.136018 2942 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.156318 3038 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.176219 1669 raft_consensus.cc:1226] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:17.174526 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3404667de13461aa4b3e79e8d6a64ee" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:17.187669 3038 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.174543 2805 raft_consensus.cc:3032] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:17.191911 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.188694 2805 raft_consensus.cc:729] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:17.200241 1741 raft_consensus.cc:1226] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260619 21:16:17.202095 1634 leader_election.cc:302] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.206570 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:17.207233 1815 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:17.165170 2860 consensus_queue.cc:1225] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer d53146739e3240239c0a3d5ea90e07fe log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260619 21:16:17.208305 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet d80394dabe604f8898e7b24149fc624f
W20260619 21:16:17.211133 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 49892.2us
I20260619 21:16:17.216144 1782 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.191777 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet 246c3d015ec242eca8bfaf5a076692b1 with delete_type TABLET_DATA_DELETED (Replaced by 655f9756924a470b8654421a17c9322e at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.222769 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:17.225210 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 60149.8us
I20260619 21:16:17.199245 2421 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:17.232779 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.232699 2421 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.233489 3045 tablet_replica.cc:330] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: stopping tablet replica
I20260619 21:16:17.233636 1810 raft_consensus.cc:2454] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:17.234079 2928 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:17.240701 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.247737 1667 raft_consensus.cc:2454] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:17.209056 1661 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:17.221576 2861 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
@ 0x7f719bd65d85 testing::TestSuite::Run() at ??:0
I20260619 21:16:17.263458 2861 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.246258 2928 raft_consensus.cc:2726] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:17.240543 2726 consensus_queue.cc:1046] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260619 21:16:17.197782 2942 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.266150 2726 raft_consensus.cc:3032] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:17.266520 2726 raft_consensus.cc:729] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:17.267117 2942 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.267997 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet 692d38d1f19c468eae7077dde218a62f
I20260619 21:16:17.246966 1710 leader_election.cc:302] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:17.269951 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 43896.8us
W20260619 21:16:17.271001 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 59003.3us
I20260619 21:16:17.250506 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:16:17.254995 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: Tablet deleted: Replaced by 21131e98f1f94e8cac1998e6c32aa089 at 2026-06-19 21:16:12 UTC
I20260619 21:16:17.271972 1782 leader_election.cc:302] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.258632 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa69d158f2b471e8638824d3d9c757b with delete_type TABLET_DATA_DELETED (Replaced by 7f05cdc19f4e420880bab15773a59709 at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.258733 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.239108 2421 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.275159 1710 leader_election.cc:302] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:17.277024 1621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Sending DeleteTablet for 3 replicas of tablet bb9b500bf9a24148b2fbcd762b549521
I20260619 21:16:17.173156 2902 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.289160 2861 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.267366 2726 consensus_queue.cc:260] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.291283 2726 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:17.293944 3038 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.301353 2716 raft_consensus.cc:2726] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:17.301424 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.302754 2981 raft_consensus.cc:3032] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:17.303252 2981 raft_consensus.cc:729] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:17.304278 2806 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:17.311058 1782 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.312614 1782 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.284292 2928 raft_consensus.cc:2726] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260619 21:16:17.285673 2942 raft_consensus.cc:2726] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:17.316828 1665 raft_consensus.cc:1261] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260619 21:16:17.321220 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 50628.7us
I20260619 21:16:17.309829 2683 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.325322 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet b249d6b40f6943e3bb7235b7df99e1b1 with delete_type TABLET_DATA_DELETED (Replaced by 016af7231f1345d3b13158794ac82632 at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.325559 1740 raft_consensus.cc:1261] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:17.327097 2683 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.329392 2602 raft_consensus.cc:2726] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:17.329690 3029 raft_consensus.cc:3032] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 LEADER]: Stepping down as leader of term 6
I20260619 21:16:17.330132 3029 raft_consensus.cc:729] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:17.331058 3029 consensus_queue.cc:260] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.332311 3029 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:17.332558 2684 leader_election.cc:288] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.334345 1782 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.315443 2860 raft_consensus.cc:2726] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:17.342044 2693 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:17.342734 2693 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.344377 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.344866 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 21131e98f1f94e8cac1998e6c32aa089 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260619 21:16:17.345918 3048 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:17.346103 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 95675.7us
I20260619 21:16:17.346575 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21131e98f1f94e8cac1998e6c32aa089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:17.346501 3048 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.348570 3048 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:17.338721 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 136703us
I20260619 21:16:17.350646 1815 raft_consensus.cc:2454] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:17.274530 2805 consensus_queue.cc:260] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.357182 2865 raft_consensus.cc:2781] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:17.357441 1801 tablet_service.cc:1514] Processing DeleteTablet for tablet bb9b500bf9a24148b2fbcd762b549521 with delete_type TABLET_DATA_DELETED (Replaced by 96effdfbfd2640ce8ee752e99d252d9e at 2026-06-19 21:16:14 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.357609 2865 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:17.358011 2865 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:17.358317 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:17.358956 1661 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:17.359072 1806 tablet_service.cc:1514] Processing DeleteTablet for tablet 692d38d1f19c468eae7077dde218a62f with delete_type TABLET_DATA_DELETED (Replaced by 35cdbd73611d4dcd915a0971b48e3d49 at 2026-06-19 21:16:14 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.360291 1634 leader_election.cc:302] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.352469 2942 consensus_queue.cc:1046] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260619 21:16:17.361922 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet d80394dabe604f8898e7b24149fc624f with delete_type TABLET_DATA_DELETED (Replaced by ae65b717a68f436586f575875021805b at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:44456
I20260619 21:16:17.362475 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.363152 1810 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:17.365545 2984 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.366941 2805 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:17.367146 1813 raft_consensus.cc:1261] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260619 21:16:17.367508 2984 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.368286 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.368459 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:17.369246 1810 raft_consensus.cc:2379] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 4: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:17.304389 2981 consensus_queue.cc:260] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.361857 2942 raft_consensus.cc:3032] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:17.384660 1663 raft_consensus.cc:1226] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:17.370924 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.352306 3048 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.382699 3027 consensus_queue.cc:1035] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260619 21:16:17.383466 2851 consensus_queue.cc:1046] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260619 21:16:17.383100 2979 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
W20260619 21:16:17.400564 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 47659.8us
I20260619 21:16:17.401396 1710 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:17.402127 2716 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.402602 2716 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.403992 1710 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:17.405287 2784 raft_consensus.cc:2726] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260619 21:16:17.406545 2784 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:17.404758 2716 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.409368 2716 raft_consensus.cc:2726] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260619 21:16:17.410166 2784 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.379026 2860 raft_consensus.cc:2726] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 21:16:17.410276 2987 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.411676 2987 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.413936 2981 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:17.414181 2987 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:17.415678 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.398640 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.416890 1661 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 6.
I20260619 21:16:17.418282 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:17.418601 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 58379us
I20260619 21:16:17.419272 1661 raft_consensus.cc:2454] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:17.421270 1669 raft_consensus.cc:2396] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260619 21:16:17.416419 1665 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:17.397163 3048 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.380044 2902 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.431764 1667 raft_consensus.cc:1226] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [term 2 LEADER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:17.430887 1663 raft_consensus.cc:1261] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:17.389072 2942 raft_consensus.cc:729] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:17.430522 2984 raft_consensus.cc:2726] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:17.382311 2693 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:17.436584 3051 log.cc:924] Time spent T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.099s user 0.000s sys 0.001s
I20260619 21:16:17.438688 1634 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.439491 2979 raft_consensus.cc:3032] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:17.439844 2979 raft_consensus.cc:729] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:17.440613 1635 leader_election.cc:302] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:17.440680 2979 consensus_queue.cc:260] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.442328 2979 raft_consensus.cc:3037] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:17.443459 2421 raft_consensus.cc:2781] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:17.444021 2421 raft_consensus.cc:686] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:17.446622 1635 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:17.447409 2984 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:17.447795 2984 raft_consensus.cc:482] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:17.448128 2984 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:17.449020 2942 consensus_queue.cc:260] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.450543 2734 consensus_queue.cc:1035] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260619 21:16:17.450601 3048 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.453306 1804 tablet_service.cc:1467] Processing CreateTablet for tablet 6ae495cfaaca419c8611fe233762a02a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260619 21:16:17.453718 3038 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:17.460426 1661 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 5. Current term is 6. Ops: []
I20260619 21:16:17.462015 2942 raft_consensus.cc:3037] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:17.462262 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:17.462251 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 127993us
I20260619 21:16:17.463733 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ae495cfaaca419c8611fe233762a02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:17.467900 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:17.468549 1810 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:17.469437 1782 leader_election.cc:398] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5.
I20260619 21:16:17.469813 1782 leader_election.cc:401] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260619 21:16:17.470412 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.472389 2980 consensus_queue.cc:1035] T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:17.476028 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.415096 2784 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.494583 2625 consensus_queue.cc:1046] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260619 21:16:17.495415 2980 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.495649 2775 raft_consensus.cc:3032] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:17.495518 1801 tablet_service.cc:1467] Processing CreateTablet for tablet 84b6bbbb60204347824250a8d71e6d3c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260619 21:16:17.495885 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1764e19eb06140f8bbdb1c3aab252d9a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.496052 2775 raft_consensus.cc:729] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:17.495983 2980 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.497287 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.497082 2775 consensus_queue.cc:260] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.497933 1742 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:17.498389 2980 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.495529 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:17.499178 3042 raft_consensus.cc:2229] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:16:17.499868 3042 raft_consensus.cc:2258] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:16:17.502522 1634 leader_election.cc:302] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.507036 1782 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.509964 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.510779 1740 raft_consensus.cc:2379] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 6: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:17.488695 2734 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.513382 1742 raft_consensus.cc:1261] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260619 21:16:17.513541 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.514595 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:17.517221 1739 raft_consensus.cc:1261] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:17.512408 3027 consensus_queue.cc:1035] T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260619 21:16:17.518965 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.488740 2851 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637) with higher term. Message: Invalid argument: T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5.
I20260619 21:16:17.485635 2861 consensus_queue.cc:1046] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260619 21:16:17.454991 2421 consensus_queue.cc:237] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.521898 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.528576 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.527563 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.528859 1815 raft_consensus.cc:3037] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:17.529412 1860 tablet_bootstrap.cc:492] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:17.523313 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.535290 1814 raft_consensus.cc:3037] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 21:16:17.533012 1742 raft_consensus.cc:2454] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:17.537801 1782 leader_election.cc:302] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.540315 1816 raft_consensus.cc:1226] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260619 21:16:17.527231 2625 consensus_queue.cc:1035] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260619 21:16:17.520500 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b6bbbb60204347824250a8d71e6d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:17.547235 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.548035 1741 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:17.549259 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.491580 2716 consensus_queue.cc:1035] T f9c4f216cdd7450b9659feeea5fd2bf9 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260619 21:16:17.514288 2936 raft_consensus.cc:2726] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 21:16:17.553495 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.521732 2930 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.559232 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.561497 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:17.536237 1634 raft_consensus.cc:2670] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260619 21:16:17.520040 2734 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.527748 2931 consensus_queue.cc:1035] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
I20260619 21:16:17.530186 2861 raft_consensus.cc:3032] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:17.497848 1806 tablet_service.cc:1467] Processing CreateTablet for tablet 18756ff2e1e84eb5b7139ecabf689d6a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260619 21:16:17.605235 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.632506 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.645256 1810 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 6.
I20260619 21:16:17.689754 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.562794 1813 raft_consensus.cc:1226] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 1. Current term is 2. Ops: []
I20260619 21:16:17.529619 2805 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:17.521708 2977 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260619 21:16:17.626669 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.693327 1809 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:17.681828 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:17.683187 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 180736us
I20260619 21:16:17.684351 1817 raft_consensus.cc:2454] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:17.680001 1738 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
W20260619 21:16:17.604821 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 44610.2us
I20260619 21:16:17.629490 1727 tablet_service.cc:1514] Processing DeleteTablet for tablet 1764e19eb06140f8bbdb1c3aab252d9a with delete_type TABLET_DATA_DELETED (Replaced by 21131e98f1f94e8cac1998e6c32aa089 at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.528206 2784 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:17.676563 3061 log.cc:924] Time spent T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260619 21:16:17.573104 2672 raft_consensus.cc:2726] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260619 21:16:17.614650 3059 log.cc:924] Time spent T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260619 21:16:17.572299 3060 log.cc:924] Time spent T c8a530b7f9264d519f65caf431a0a478 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260619 21:16:17.695654 1782 leader_election.cc:302] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.705353 1634 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.712972 1812 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
I20260619 21:16:17.706329 1783 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:17.648958 1726 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b2adcb38244b60bdd7ae5d5642f641 with delete_type TABLET_DATA_DELETED (Replaced by 18756ff2e1e84eb5b7139ecabf689d6a at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.716603 1809 raft_consensus.cc:1261] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260619 21:16:17.717072 1809 pending_rounds.cc:76] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Aborting all ops after (but not including) 2
I20260619 21:16:17.715894 1818 raft_consensus.cc:1226] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:17.629287 2923 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:17.708559 1664 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:17.710541 1729 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae81d060b714ae98341d6b5e2669f3f with delete_type TABLET_DATA_DELETED (Replaced by 331f6581b0ac4c3bbab86ecc34ececfb at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.649466 1734 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae5f9625019495d85bc13c82d7f621b with delete_type TABLET_DATA_DELETED (Replaced by 6ae495cfaaca419c8611fe233762a02a at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.719700 1782 leader_election.cc:302] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.555440 2851 consensus_queue.cc:1035] T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.154s
I20260619 21:16:17.715229 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.675467 1730 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dea6228e49c44b9889aa709158ef557 with delete_type TABLET_DATA_DELETED (Replaced by 84b6bbbb60204347824250a8d71e6d3c at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.684321 2861 raft_consensus.cc:729] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:17.717320 1809 pending_rounds.cc:97] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260619 21:16:17.726532 1813 raft_consensus.cc:1226] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260619 21:16:17.726101 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.571455 2920 consensus_queue.cc:1035] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.228s
I20260619 21:16:17.726951 1809 raft_consensus.cc:2866] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260619 21:16:17.727419 1660 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:17.729867 1635 leader_election.cc:302] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.699908 2860 consensus_queue.cc:1035] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260619 21:16:17.733831 1816 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:17.741025 1800 tablet_service.cc:1467] Processing CreateTablet for tablet a7a69751ebea42d5950577710f9b1301 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260619 21:16:17.741257 1805 tablet_service.cc:1467] Processing CreateTablet for tablet 655f9756924a470b8654421a17c9322e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260619 21:16:17.718451 3038 raft_consensus.cc:2726] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:17.706236 2805 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.706384 2930 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.751672 1726 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 with delete_type TABLET_DATA_DELETED (Replaced by a7a69751ebea42d5950577710f9b1301 at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.724200 2994 raft_consensus.cc:2781] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:17.756068 1634 leader_election.cc:302] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.720561 2975 consensus_queue.cc:1046] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260619 21:16:17.757138 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18756ff2e1e84eb5b7139ecabf689d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:17.723919 2684 consensus_queue.cc:1035] T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260619 21:16:17.781083 2805 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.781687 2902 raft_consensus.cc:2726] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:17.781797 3042 ts_tablet_manager.cc:1905] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 21:16:17.783285 1634 leader_election.cc:302] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.759688 2994 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260619 21:16:17.801306 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 45542.5us
I20260619 21:16:17.765460 3038 raft_consensus.cc:2781] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260619 21:16:17.817936 1634 leader_election.cc:302] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:17.723659 2923 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.825274 1634 leader_election.cc:302] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.826771 1634 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:17.770684 1734 tablet_service.cc:1514] Processing DeleteTablet for tablet 246c3d015ec242eca8bfaf5a076692b1 with delete_type TABLET_DATA_DELETED (Replaced by 655f9756924a470b8654421a17c9322e at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.779115 2775 raft_consensus.cc:3037] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:17.822234 2994 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:17.761507 2693 consensus_queue.cc:1035] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260619 21:16:17.841878 1810 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:17.844498 2900 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:17.845057 2900 raft_consensus.cc:686] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:17.845660 1802 tablet_service.cc:1467] Processing CreateTablet for tablet 016af7231f1345d3b13158794ac82632 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260619 21:16:17.845968 2900 consensus_queue.cc:237] T cd1e5b0f2736497bb5ea639a3071f8a2 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.837098 3038 raft_consensus.cc:686] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [term 6 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:17.847221 1726 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa69d158f2b471e8638824d3d9c757b with delete_type TABLET_DATA_DELETED (Replaced by 7f05cdc19f4e420880bab15773a59709 at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.847720 3038 consensus_queue.cc:237] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.849112 1727 tablet_service.cc:1514] Processing DeleteTablet for tablet b249d6b40f6943e3bb7235b7df99e1b1 with delete_type TABLET_DATA_DELETED (Replaced by 016af7231f1345d3b13158794ac82632 at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.850916 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.854732 1803 tablet_service.cc:1467] Processing CreateTablet for tablet 7f05cdc19f4e420880bab15773a59709 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260619 21:16:17.834600 2902 raft_consensus.cc:2726] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:17.848824 1654 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa69d158f2b471e8638824d3d9c757b with delete_type TABLET_DATA_DELETED (Replaced by 7f05cdc19f4e420880bab15773a59709 at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.832329 2809 raft_consensus.cc:2781] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:17.864429 1813 raft_consensus.cc:1261] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260619 21:16:17.826140 2805 raft_consensus.cc:2726] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:17.865381 1782 leader_election.cc:302] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.834692 1657 tablet_service.cc:1514] Processing DeleteTablet for tablet 246c3d015ec242eca8bfaf5a076692b1 with delete_type TABLET_DATA_DELETED (Replaced by 655f9756924a470b8654421a17c9322e at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.834874 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f709f3b1e904d1ba8bf5592d77cc8d6 with delete_type TABLET_DATA_DELETED (Replaced by a7a69751ebea42d5950577710f9b1301 at 2026-06-19 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.863636 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7a69751ebea42d5950577710f9b1301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:17.869858 1661 raft_consensus.cc:1226] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 3. Current term is 4. Ops: []
I20260619 21:16:17.870581 1782 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.872026 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:17.869331 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:17.796042 2861 consensus_queue.cc:260] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:17.865842 2809 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:17.877842 1740 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:17.870187 2734 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:17.903396 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:17.901682 1727 tablet_service.cc:1514] Processing DeleteTablet for tablet d80394dabe604f8898e7b24149fc624f with delete_type TABLET_DATA_DELETED (Replaced by ae65b717a68f436586f575875021805b at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.872709 2851 raft_consensus.cc:2781] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:17.918717 1635 leader_election.cc:302] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:17.919945 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet d80394dabe604f8898e7b24149fc624f with delete_type TABLET_DATA_DELETED (Replaced by ae65b717a68f436586f575875021805b at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:17.920241 2734 consensus_queue.cc:1035] T b249d6b40f6943e3bb7235b7df99e1b1 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260619 21:16:17.921993 2734 raft_consensus.cc:2726] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 21:16:17.924304 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:17.931243 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 60712.3us
I20260619 21:16:17.907333 2809 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:17.839331 1808 tablet_service.cc:1467] Processing CreateTablet for tablet 331f6581b0ac4c3bbab86ecc34ececfb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260619 21:16:17.906939 2977 raft_consensus.cc:2781] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 21:16:17.941851 1727 tablet_service.cc:1514] Processing DeleteTablet for tablet 692d38d1f19c468eae7077dde218a62f with delete_type TABLET_DATA_DELETED (Replaced by 35cdbd73611d4dcd915a0971b48e3d49 at 2026-06-19 21:16:14 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.925770 1654 tablet_service.cc:1514] Processing DeleteTablet for tablet b249d6b40f6943e3bb7235b7df99e1b1 with delete_type TABLET_DATA_DELETED (Replaced by 016af7231f1345d3b13158794ac82632 at 2026-06-19 21:16:13 UTC) from {username='slave'} at 127.0.0.1:38114
W20260619 21:16:17.926277 2865 consensus_meta.cc:287] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.568s user 0.011s sys 0.001s
I20260619 21:16:17.926203 2684 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:17.935948 2851 raft_consensus.cc:686] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:17.973090 1727 tablet_service.cc:1514] Processing DeleteTablet for tablet bb9b500bf9a24148b2fbcd762b549521 with delete_type TABLET_DATA_DELETED (Replaced by 96effdfbfd2640ce8ee752e99d252d9e at 2026-06-19 21:16:14 UTC) from {username='slave'} at 127.0.0.1:35686
I20260619 21:16:17.965866 1805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655f9756924a470b8654421a17c9322e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:17.982780 1807 tablet_service.cc:1467] Processing CreateTablet for tablet 96effdfbfd2640ce8ee752e99d252d9e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260619 21:16:17.985297 1799 tablet_service.cc:1467] Processing CreateTablet for tablet 35cdbd73611d4dcd915a0971b48e3d49 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260619 21:16:17.989885 1730 tablet_service.cc:1467] Processing CreateTablet for tablet 84b6bbbb60204347824250a8d71e6d3c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260619 21:16:17.990159 1726 tablet_service.cc:1467] Processing CreateTablet for tablet 6ae495cfaaca419c8611fe233762a02a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260619 21:16:17.991042 1734 tablet_service.cc:1467] Processing CreateTablet for tablet 18756ff2e1e84eb5b7139ecabf689d6a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260619 21:16:17.991547 1729 tablet_service.cc:1467] Processing CreateTablet for tablet 331f6581b0ac4c3bbab86ecc34ececfb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260619 21:16:17.992570 3064 log.cc:924] Time spent T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.191s user 0.000s sys 0.001s
I20260619 21:16:17.965458 1801 tablet_service.cc:1467] Processing CreateTablet for tablet ae65b717a68f436586f575875021805b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260619 21:16:17.964267 2684 raft_consensus.cc:482] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:17.927194 1563 catalog_manager.cc:5495] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 4, leader changed from d53146739e3240239c0a3d5ea90e07fe (127.0.99.2) to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 4 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:16:17.949079 3040 consensus_queue.cc:1035] T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260619 21:16:17.976905 2977 raft_consensus.cc:686] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:18.009240 2684 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:18.015612 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae65b717a68f436586f575875021805b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.017560 1563 catalog_manager.cc:5495] T c28152c520fe41e5be2905e6e3b67b8a P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 2 to 5, leader changed from d53146739e3240239c0a3d5ea90e07fe (127.0.99.2) to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 5 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:16:18.019351 1563 catalog_manager.cc:5495] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 2 to 3, leader changed from d53146739e3240239c0a3d5ea90e07fe (127.0.99.2) to 884c214bb0a7455d91701ec388635e74 (127.0.99.1). New cstate: current_term: 3 leader_uuid: "884c214bb0a7455d91701ec388635e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260619 21:16:17.964073 2865 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.021840 1662 raft_consensus.cc:1226] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:18.010910 1727 tablet_service.cc:1467] Processing CreateTablet for tablet 21131e98f1f94e8cac1998e6c32aa089 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260619 21:16:17.988579 2994 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.031250 1803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f05cdc19f4e420880bab15773a59709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.029667 2930 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.038265 1725 tablet_service.cc:1467] Processing CreateTablet for tablet 655f9756924a470b8654421a17c9322e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260619 21:16:18.038892 1726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ae495cfaaca419c8611fe233762a02a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260619 21:16:18.012981 3065 log.cc:924] Time spent T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.242s user 0.001s sys 0.000s
I20260619 21:16:18.039583 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:18.039872 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:18.042624 1743 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:18.043187 1563 catalog_manager.cc:5495] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d53146739e3240239c0a3d5ea90e07fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } }
W20260619 21:16:18.034293 3069 log.cc:924] Time spent T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.065s user 0.000s sys 0.001s
I20260619 21:16:18.010773 3046 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.048229 1783 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:18.054579 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 111805us
I20260619 21:16:18.061463 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:18.062465 1665 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
I20260619 21:16:18.063555 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:18.064240 1665 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 3
I20260619 21:16:18.066500 1664 raft_consensus.cc:1226] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 7 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260619 21:16:18.068529 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:17.993546 1732 tablet_service.cc:1467] Processing CreateTablet for tablet a7a69751ebea42d5950577710f9b1301 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260619 21:16:18.093801 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:18.094930 1744 raft_consensus.cc:3037] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:18.058923 2981 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:18.033300 2923 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:18.042337 2994 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:18.093760 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331f6581b0ac4c3bbab86ecc34ececfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.101557 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21131e98f1f94e8cac1998e6c32aa089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.049700 3027 raft_consensus.cc:2726] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:18.099577 2981 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260619 21:16:18.107643 2984 consensus_meta.cc:287] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.659s user 0.014s sys 0.000s
I20260619 21:16:18.106788 2865 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.128564 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655f9756924a470b8654421a17c9322e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.113503 2900 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.151543 1728 tablet_service.cc:1467] Processing CreateTablet for tablet 7f05cdc19f4e420880bab15773a59709 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260619 21:16:18.151857 1733 tablet_service.cc:1467] Processing CreateTablet for tablet 016af7231f1345d3b13158794ac82632 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260619 21:16:18.152758 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016af7231f1345d3b13158794ac82632. 1 dirs total, 0 dirs full, 0 dirs failed
W20260619 21:16:18.153515 3072 log.cc:924] Time spent T cfa69d158f2b471e8638824d3d9c757b P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260619 21:16:18.135875 2984 raft_consensus.cc:504] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.049476 2851 consensus_queue.cc:237] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.155454 2984 leader_election.cc:288] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.136245 2930 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:18.156407 2930 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.134531 2977 consensus_queue.cc:237] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.159464 1666 raft_consensus.cc:2379] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 4: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
@ 0x7f719bd7a875 testing::internal::UnitTestImpl::RunAllTests() at ??:0
I20260619 21:16:18.105609 3046 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.130118 2981 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:18.217687 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:18.218374 1740 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:18.220863 3075 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.229898 2942 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
W20260619 21:16:18.233800 3067 log.cc:924] Time spent T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.173s user 0.001s sys 0.000s
I20260619 21:16:18.234030 2942 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.234544 1731 tablet_service.cc:1467] Processing CreateTablet for tablet ae65b717a68f436586f575875021805b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260619 21:16:18.238922 1668 raft_consensus.cc:1226] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260619 21:16:18.241422 3040 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:18.281700 2920 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.282266 2920 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.241611 3075 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:18.328362 3077 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.104s user 0.000s sys 0.001s
I20260619 21:16:18.343202 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96effdfbfd2640ce8ee752e99d252d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.289983 3040 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.351280 2860 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.382580 1662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 622) took 1037 ms. Trace:
W20260619 21:16:18.363147 3070 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.426s user 0.000s sys 0.003s
I20260619 21:16:18.383167 1662 rpcz_store.cc:276] 0619 21:16:17.350229 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.365043 (+ 14814us) service_pool.cc:227] Handling call
0619 21:16:17.365371 (+ 328us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:18.028849 (+663478us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:18.382219 (+353370us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54000a9454. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:18.382490 (+ 271us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1509138176}
I20260619 21:16:18.393386 3029 raft_consensus.cc:3032] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Stepping down as leader of term 3
I20260619 21:16:18.400771 3029 raft_consensus.cc:729] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:18.402940 3029 consensus_queue.cc:260] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.404006 3029 raft_consensus.cc:3037] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:18.405006 2860 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.417611 1654 tablet_service.cc:1514] Processing DeleteTablet for tablet 692d38d1f19c468eae7077dde218a62f with delete_type TABLET_DATA_DELETED (Replaced by 35cdbd73611d4dcd915a0971b48e3d49 at 2026-06-19 21:16:14 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:18.417979 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet bb9b500bf9a24148b2fbcd762b549521 with delete_type TABLET_DATA_DELETED (Replaced by 96effdfbfd2640ce8ee752e99d252d9e at 2026-06-19 21:16:14 UTC) from {username='slave'} at 127.0.0.1:38114
I20260619 21:16:18.419551 1657 tablet_service.cc:1467] Processing CreateTablet for tablet 21131e98f1f94e8cac1998e6c32aa089 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260619 21:16:18.419551 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 6ae495cfaaca419c8611fe233762a02a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260619 21:16:18.422478 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21131e98f1f94e8cac1998e6c32aa089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.434108 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ae495cfaaca419c8611fe233762a02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.455380 1739 raft_consensus.cc:3032] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Stepping down as leader of term 5
I20260619 21:16:18.455890 1739 raft_consensus.cc:729] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:18.456782 1739 consensus_queue.cc:260] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.465279 1739 raft_consensus.cc:3037] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:18.478729 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 635) took 1046 ms. Trace:
I20260619 21:16:18.479244 1666 rpcz_store.cc:276] 0619 21:16:17.432279 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.455984 (+ 23705us) service_pool.cc:227] Handling call
0619 21:16:18.478441 (+1022457us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b5400332f54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:18.478620 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1582850944}
I20260619 21:16:18.481760 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cdbd73611d4dcd915a0971b48e3d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.485936 3068 tablet_replica.cc:330] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: stopping tablet replica
I20260619 21:16:18.555048 2806 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:18.555892 2806 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.560158 2806 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.561118 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:18.576866 1663 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.011s sys 0.019s
W20260619 21:16:18.577778 1663 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s user 0.012s sys 0.019s
W20260619 21:16:18.579725 1784 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.574s user 0.016s sys 0.018s
W20260619 21:16:18.580315 1784 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.575s user 0.016s sys 0.018s
I20260619 21:16:18.581734 2994 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.582368 2994 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:18.585395 1738 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s user 0.000s sys 0.000s
W20260619 21:16:18.585906 1738 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.534s user 0.000s sys 0.000s
I20260619 21:16:18.586548 2806 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:18.590263 2684 consensus_meta.cc:287] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.575s user 0.009s sys 0.000s
I20260619 21:16:18.590806 2684 raft_consensus.cc:504] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.594283 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:18.595162 2865 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.595786 2865 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.597137 1741 raft_consensus.cc:3037] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:18.598693 3068 raft_consensus.cc:2229] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260619 21:16:18.601707 3068 raft_consensus.cc:2258] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260619 21:16:18.609601 3046 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:18.610947 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 329797us
W20260619 21:16:18.620056 1782 consensus_peers.cc:487] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 -> Peer 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637): Couldn't send request to peer 884c214bb0a7455d91701ec388635e74. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260619 21:16:18.610033 2930 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:18.653398 1744 consensus_meta.cc:287] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.558s user 0.010s sys 0.000s
I20260619 21:16:18.653951 1744 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:18.610643 2861 raft_consensus.cc:3037] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:18.588588 2806 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.659090 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:18.676826 1738 raft_consensus.cc:2396] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:18.680141 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
W20260619 21:16:18.653437 3084 log.cc:924] Time spent T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.144s user 0.001s sys 0.000s
I20260619 21:16:18.619346 2987 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.615878 2994 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:18.629156 3028 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:18.708652 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 641) took 1023 ms. Trace:
I20260619 21:16:18.712198 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae5f9625019495d85bc13c82d7f621b" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:18.715219 1816 raft_consensus.cc:3037] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:18.659030 2716 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.665846 2602 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.610203 2900 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.688990 1732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7a69751ebea42d5950577710f9b1301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.665704 3049 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:18.669364 2928 raft_consensus.cc:2726] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 21:16:18.714699 1812 rpcz_store.cc:276] 0619 21:16:17.684830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.715330 (+ 30500us) service_pool.cc:227] Handling call
0619 21:16:17.721469 (+ 6139us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:17.744113 (+ 22644us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:17.744142 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:17.762263 (+ 18121us) log.cc:842] Serialized 26 byte log entry
0619 21:16:17.849945 (+ 87682us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:17.849987 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:17.850053 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:17.921520 (+ 71467us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b48000e0dd8. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a39657f1 00007f71a395c8e7 00007f71a395c77a 00007f71a3a5095f 00007f71a3a50627 00007f71a3a64b2c 00007f71a3a64ad8 00007f71a3a64a58 00007f71a3a64a13 00007f71a3a63cbc 00007f71a29e5be6
0619 21:16:18.362416 (+440896us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719e740bee 00007f719e73ff36 00007f719e73fb58 00007f719e749551 00007f719e749509 00007f719e749499
0619 21:16:18.708431 (+346015us) raft_consensus.cc:1671] finished
0619 21:16:18.708452 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:18.708585 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11348,"raft.queue_time_us":11949,"raft.run_cpu_time_us":1726,"raft.run_wall_time_us":179903,"spinlock_wait_cycles":16433280,"thread_start_us":42414,"threads_started":1,"wal-append.queue_time_us":100357}
I20260619 21:16:18.731731 1860 tablet_bootstrap.cc:654] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:16:18.740437 1665 consensus_meta.cc:287] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.676s user 0.014s sys 0.000s
W20260619 21:16:18.743849 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:18.671283 2734 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:18.743513 1665 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:18.738524 3040 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:18.802989 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:18.803082 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:18.743945 1743 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.005s sys 0.028s
W20260619 21:16:18.805931 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 83532.6us
W20260619 21:16:18.817907 1743 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.577s user 0.006s sys 0.028s
I20260619 21:16:18.817690 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:18.815963 1816 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:18.821557 3045 raft_consensus.cc:2229] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260619 21:16:18.733835 2684 leader_election.cc:288] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:18.802345 3064 log.cc:924] Time spent T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260619 21:16:18.823982 1810 raft_consensus.cc:2379] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:18.818437 1710 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:18.828039 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:18.831130 1653 tablet_service.cc:1467] Processing CreateTablet for tablet 331f6581b0ac4c3bbab86ecc34ececfb (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260619 21:16:18.831487 1652 tablet_service.cc:1467] Processing CreateTablet for tablet 84b6bbbb60204347824250a8d71e6d3c (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260619 21:16:18.736558 2716 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.836006 2716 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.842104 1650 tablet_service.cc:1467] Processing CreateTablet for tablet ae65b717a68f436586f575875021805b (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260619 21:16:18.842379 1659 tablet_service.cc:1467] Processing CreateTablet for tablet 7f05cdc19f4e420880bab15773a59709 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260619 21:16:18.867959 1658 tablet_service.cc:1467] Processing CreateTablet for tablet 18756ff2e1e84eb5b7139ecabf689d6a (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260619 21:16:18.822625 3079 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.869369 3079 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.869571 1656 tablet_service.cc:1467] Processing CreateTablet for tablet 016af7231f1345d3b13158794ac82632 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260619 21:16:18.871641 3079 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:18.872081 1655 tablet_service.cc:1467] Processing CreateTablet for tablet 655f9756924a470b8654421a17c9322e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260619 21:16:18.809353 2942 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.886866 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:18.813580 3075 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:18.822340 3065 log.cc:924] Time spent T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20260619 21:16:18.918438 2920 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.919376 3045 raft_consensus.cc:2258] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260619 21:16:18.888386 1804 consensus_meta.cc:287] T 6ae495cfaaca419c8611fe233762a02a P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.026s user 0.025s sys 0.024s
W20260619 21:16:18.807072 1708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.99.1:45637 timed out after 1.708s (SENT) [suppressed 14 similar messages]
I20260619 21:16:18.809288 2930 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:18.875252 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:18.923314 1744 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:18.742607 2602 raft_consensus.cc:504] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.895920 1651 tablet_service.cc:1467] Processing CreateTablet for tablet a7a69751ebea42d5950577710f9b1301 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260619 21:16:18.731439 3028 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.908738 1816 raft_consensus.cc:2396] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:18.720467 2987 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.840119 2860 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:18.806164 2734 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:18.982856 1812 raft_consensus.cc:1261] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:18.806313 3049 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:18.928471 3091 log.cc:924] Time spent T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260619 21:16:18.902428 2984 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:18.922461 1804 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 72) took 1470 ms. Trace:
I20260619 21:16:19.002274 1818 raft_consensus.cc:1261] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260619 21:16:18.925048 2942 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:18.993736 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae65b717a68f436586f575875021805b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:18.993183 1653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331f6581b0ac4c3bbab86ecc34ececfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.015026 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 648) took 1085 ms. Trace:
I20260619 21:16:19.006496 1804 rpcz_store.cc:276] 0619 21:16:17.452093 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.452240 (+ 147us) service_pool.cc:227] Handling call
0619 21:16:17.463112 (+ 10872us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.463232 (+ 120us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:17.796366 (+333134us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:18.921761 (+1125395us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:18.922374 (+ 613us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:19.017045 1813 rpcz_store.cc:276] 0619 21:16:17.929574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.929807 (+ 233us) service_pool.cc:227] Handling call
0619 21:16:17.944886 (+ 15079us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:18.098931 (+154045us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:18.098970 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:18.110242 (+ 11272us) log.cc:842] Serialized 26 byte log entry
0619 21:16:18.627320 (+517078us) raft_consensus.cc:1631] Marking committed up to 0
0619 21:16:18.627368 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:18.627474 (+ 106us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:18.836448 (+208974us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b2400410fa8. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719b9eecf4 00007f719b9ecdfb 00007f719ba7b559 00007f719ba81495 00007f719ba81362 00007f719ba813ee 00007f719b4bf5db 00007f71a39e4dda 00007f719ba812fc 00007f719ba739bd
0619 21:16:19.006819 (+170371us) raft_consensus.cc:1671] finished
0619 21:16:19.006858 (+ 39us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.014938 (+ 8080us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9321,"raft.queue_time_us":56242,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":11817,"spinlock_wait_cycles":9121152,"thread_start_us":486813,"threads_started":2,"wal-append.queue_time_us":356948}
I20260619 21:16:18.984557 2900 leader_election.cc:288] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:18.983682 2930 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.010192 3027 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:19.037694 1783 leader_election.cc:302] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:19.045114 3083 raft_consensus.cc:2781] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader election won for term 3
W20260619 21:16:19.046005 3088 log.cc:924] Time spent T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.227s user 0.000s sys 0.000s
I20260619 21:16:19.046973 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 682) took 1540 ms. Trace:
I20260619 21:16:19.007661 2984 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.052131 3092 log.cc:924] Time spent T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.070s user 0.001s sys 0.001s
I20260619 21:16:19.049995 1737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 613) took 1118 ms. Trace:
I20260619 21:16:19.047377 1736 rpcz_store.cc:276] 0619 21:16:17.506076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.535959 (+ 29883us) service_pool.cc:227] Handling call
0619 21:16:17.677652 (+141693us) raft_consensus.cc:1457] Updating replica for 2 ops
0619 21:16:17.771792 (+ 94140us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:17.771815 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0619 21:16:17.785223 (+ 13408us) log.cc:842] Serialized 52 byte log entry
0619 21:16:18.409704 (+624481us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18000fda20. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e745411 00007f719ea925cf
0619 21:16:18.739359 (+329655us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:18.739400 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:18.926413 (+187013us) spinlock_profiling.cc:243] Waited 93.7 ms on lock 0x7b54002ee0d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:18.926913 (+ 500us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:19.046696 (+119783us) raft_consensus.cc:1671] finished
0619 21:16:19.046724 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.046910 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4478,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":11817,"spinlock_wait_cycles":240877824,"thread_start_us":329467,"threads_started":1,"wal-append.queue_time_us":1164329}
I20260619 21:16:19.015360 2942 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.042017 2865 leader_election.cc:288] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.058758 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b6bbbb60204347824250a8d71e6d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.053650 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331f6581b0ac4c3bbab86ecc34ececfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.050149 3083 raft_consensus.cc:686] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Becoming Leader. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:19.094023 1635 leader_election.cc:302] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:19.091888 1783 leader_election.cc:302] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:19.059368 2806 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:19.088855 2809 consensus_meta.cc:287] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.128s user 0.017s sys 0.036s
W20260619 21:16:19.090664 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 416394us
W20260619 21:16:19.098238 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 279808us
I20260619 21:16:19.089224 3045 ts_tablet_manager.cc:1905] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 21:16:19.101989 3068 ts_tablet_manager.cc:1905] T 1764e19eb06140f8bbdb1c3aab252d9a P d53146739e3240239c0a3d5ea90e07fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 21:16:19.043469 3027 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.089254 2775 consensus_meta.cc:287] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.254s user 0.017s sys 0.028s
I20260619 21:16:19.078773 2980 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:19.111976 3102 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:19.143946 1741 consensus_meta.cc:287] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.544s user 0.017s sys 0.009s
I20260619 21:16:19.191581 1741 raft_consensus.cc:2454] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:19.122663 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.181725 1710 leader_election.cc:302] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:19.193396 1738 raft_consensus.cc:1226] T deaff01bc5d54f808990dbcbaf46c0b1 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260619 21:16:19.164625 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 70710.8us
I20260619 21:16:19.125224 2602 leader_election.cc:288] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:19.122319 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.222452 1744 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:19.206805 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.271078 1735 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:19.154093 1814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 654) took 1619 ms. Trace:
I20260619 21:16:19.193635 1736 raft_consensus.cc:2454] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 4.
W20260619 21:16:19.226824 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 45189.9us
I20260619 21:16:19.275619 1710 leader_election.cc:302] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:19.227322 1743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 701) took 1152 ms. Trace:
I20260619 21:16:19.277405 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:19.228000 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 420877us
I20260619 21:16:19.101384 2865 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.128193 2930 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.137959 2979 raft_consensus.cc:2781] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260619 21:16:19.137511 2734 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.238023 1663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 601) took 1180 ms. Trace:
W20260619 21:16:19.301118 1740 consensus_meta.cc:287] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.079s user 0.012s sys 0.009s
I20260619 21:16:19.301522 1740 raft_consensus.cc:2454] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:19.243373 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae65b717a68f436586f575875021805b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.243953 3083 consensus_queue.cc:237] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.247983 2861 consensus_meta.cc:287] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.579s user 0.011s sys 0.001s
I20260619 21:16:19.250286 2984 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.324355 2984 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.324457 3105 consensus_queue.cc:1046] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260619 21:16:19.324973 2984 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.181237 2809 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.240691 2775 raft_consensus.cc:2781] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:19.212648 2806 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.137539 3028 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:19.224061 3074 raft_consensus.cc:482] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.294384 1805 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 77) took 1565 ms. Trace:
I20260619 21:16:19.296520 1802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 79) took 1517 ms. Trace:
I20260619 21:16:19.398820 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:19.301723 1783 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:19.302127 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35690 (request call id 616) took 1141 ms. Trace:
W20260619 21:16:19.303099 1739 consensus_meta.cc:287] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.829s user 0.017s sys 0.003s
I20260619 21:16:19.412088 1739 raft_consensus.cc:1261] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260619 21:16:19.061480 1737 rpcz_store.cc:276] 0619 21:16:17.931930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.936304 (+ 4374us) service_pool.cc:227] Handling call
0619 21:16:17.995888 (+ 59584us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:18.041118 (+ 45230us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:18.046156 (+ 5038us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:18.113368 (+ 67212us) log.cc:842] Serialized 26 byte log entry
0619 21:16:18.653070 (+539702us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:18.664085 (+ 11015us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:18.690301 (+ 26216us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:19.047167 (+356866us) raft_consensus.cc:1671] finished
0619 21:16:19.049712 (+ 2545us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.049915 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2804,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":122240,"thread_start_us":406488,"threads_started":1,"wal-append.queue_time_us":582346}
I20260619 21:16:19.277096 1743 rpcz_store.cc:276] 0619 21:16:18.075062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.075237 (+ 175us) service_pool.cc:227] Handling call
0619 21:16:18.236091 (+160854us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5eb7c 00007f719bb5e73b 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:18.236305 (+ 214us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:18.236537 (+ 232us) raft_consensus.cc:1520] Early marking committed up to index 4
0619 21:16:18.838722 (+602185us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.204586 (+365864us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.226934 (+ 22348us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:19.226990 (+ 56us) raft_consensus.cc:1631] Marking committed up to 4
0619 21:16:19.227030 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:19.227098 (+ 68us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.227240 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":60793,"prepare.queue_time_us":49121,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":101,"raft.queue_time_us":25582,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":10460,"spinlock_wait_cycles":6726912,"thread_start_us":886816,"threads_started":2,"wal-append.queue_time_us":423999}
W20260619 21:16:19.313035 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 118506us
I20260619 21:16:19.272608 1814 rpcz_store.cc:276] 0619 21:16:17.534692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.534805 (+ 113us) service_pool.cc:227] Handling call
0619 21:16:17.535124 (+ 319us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:17.934033 (+398909us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:17.934102 (+ 69us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:17.935117 (+ 1015us) log.cc:842] Serialized 26 byte log entry
0619 21:16:18.443954 (+508837us) spinlock_profiling.cc:243] Waited 9.47 ms on lock 0x7b1800330a60. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e745411 00007f719ea925cf
0619 21:16:18.630166 (+186212us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:18.663274 (+ 33108us) log.cc:842] Serialized 14 byte log entry
0619 21:16:18.665949 (+ 2675us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:18.666026 (+ 77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:19.153690 (+487664us) raft_consensus.cc:1671] finished
0619 21:16:19.153819 (+ 129us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.154008 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44772,"prepare.queue_time_us":28986,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":13869,"raft.queue_time_us":16930,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":27251,"spinlock_wait_cycles":21785984,"thread_start_us":186029,"threads_started":1,"wal-append.queue_time_us":798635}
I20260619 21:16:19.254381 3080 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:19.259757 2900 raft_consensus.cc:2781] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:19.212598 2980 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.397616 3046 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.592s user 0.011s sys 0.006s
I20260619 21:16:19.298761 3103 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:19.303504 2975 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.609s user 0.009s sys 0.017s
W20260619 21:16:19.393178 1667 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.025s sys 0.031s
W20260619 21:16:19.394137 1661 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.676s user 0.007s sys 0.088s
I20260619 21:16:19.299994 2979 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:19.134500 2942 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:19.397488 1782 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.589s user 0.008s sys 0.008s
I20260619 21:16:19.322294 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016af7231f1345d3b13158794ac82632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.303670 2716 raft_consensus.cc:2781] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:19.386070 2775 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:19.396912 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.413280 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.299233 2865 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.443822 1635 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:19.416435 3087 log.cc:924] Time spent T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.595s user 0.000s sys 0.002s
I20260619 21:16:19.417294 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:19.214910 3104 consensus_queue.cc:1046] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.231s
I20260619 21:16:19.418543 3027 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:19.421169 3096 log.cc:924] Time spent T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.434s user 0.002s sys 0.001s
W20260619 21:16:19.422981 3086 log.cc:924] Time spent T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.521s user 0.001s sys 0.001s
I20260619 21:16:19.386068 2930 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:19.410880 1740 rpcz_store.cc:276] 0619 21:16:18.160877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.163172 (+ 2295us) service_pool.cc:227] Handling call
0619 21:16:19.302052 (+1138880us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:19.427772 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.212777 3102 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.432076 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 130441us
I20260619 21:16:19.432548 1660 raft_consensus.cc:1226] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:19.321673 2920 raft_consensus.cc:2741] T 5ae5f9625019495d85bc13c82d7f621b P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260619 21:16:19.321467 2734 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:19.300522 1663 rpcz_store.cc:276] 0619 21:16:18.057944 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.069977 (+ 12033us) service_pool.cc:227] Handling call
0619 21:16:18.070437 (+ 460us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:18.070896 (+ 459us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:18.070937 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:18.071499 (+ 562us) log.cc:842] Serialized 26 byte log entry
0619 21:16:18.578312 (+506813us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:18.703011 (+124699us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.228573 (+525562us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:19.228679 (+ 106us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:19.228737 (+ 58us) raft_consensus.cc:1671] finished
0619 21:16:19.228755 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.237933 (+ 9178us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40178,"prepare.queue_time_us":261659,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":129041,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":38732,"thread_start_us":1129058,"threads_started":3,"wal-append.queue_time_us":366927}
I20260619 21:16:19.434974 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:19.602026 1662 raft_consensus.cc:2396] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:19.602108 1743 raft_consensus.cc:2454] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:19.416673 3105 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:19.391993 2806 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.358675 3049 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:19.478549 1661 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.761s user 0.007s sys 0.089s
W20260619 21:16:19.478655 1782 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.670s user 0.009s sys 0.009s
I20260619 21:16:19.612340 1727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35686 (request call id 71) took 1615 ms. Trace:
W20260619 21:16:19.424898 1799 consensus_meta.cc:287] T 35cdbd73611d4dcd915a0971b48e3d49 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.505s user 0.004s sys 0.009s
I20260619 21:16:19.413652 1665 raft_consensus.cc:1261] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 7 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260619 21:16:19.503875 1634 leader_election.cc:302] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:19.592509 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 73833.5us
I20260619 21:16:19.565428 1669 raft_consensus.cc:2454] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 2.
W20260619 21:16:19.516860 1667 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.775s user 0.025s sys 0.040s
I20260619 21:16:19.515105 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:19.499528 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:19.623031 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:19.623567 1668 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 5.
I20260619 21:16:19.396628 2861 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:19.518332 2942 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
W20260619 21:16:19.485985 1742 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.731s user 0.014s sys 0.017s
W20260619 21:16:19.630645 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 126844us
I20260619 21:16:19.637346 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:19.638995 1742 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.893s user 0.016s sys 0.020s
I20260619 21:16:19.643007 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 705) took 1476 ms. Trace:
I20260619 21:16:19.600831 1783 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:19.601128 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 319226us
I20260619 21:16:19.443008 2931 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260619 21:16:19.426836 2684 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:19.498493 3109 log.cc:924] Time spent T 2cb9bc6419d14ea8b9834eea6fafc54a P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260619 21:16:19.540071 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18756ff2e1e84eb5b7139ecabf689d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.404690 3074 raft_consensus.cc:504] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.470516 3028 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.434592 2900 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:19.514642 3121 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.522810 2979 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:19.443898 2923 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:19.479192 2975 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.797s user 0.016s sys 0.018s
I20260619 21:16:19.515268 2716 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:19.519608 2775 raft_consensus.cc:3037] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
W20260619 21:16:19.602455 3046 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.795s user 0.011s sys 0.019s
I20260619 21:16:19.470054 2977 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.655979 2931 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:19.435110 1801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 80) took 1564 ms. Trace:
I20260619 21:16:19.661098 1663 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:19.664692 1808 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 75) took 1950 ms. Trace:
I20260619 21:16:19.643380 1742 rpcz_store.cc:276] 0619 21:16:18.166478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.166595 (+ 117us) service_pool.cc:227] Handling call
0619 21:16:18.218160 (+ 51565us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:18.218382 (+ 222us) raft_consensus.cc:1520] Early marking committed up to index 2
0619 21:16:18.713138 (+494756us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.642545 (+929407us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:19.642619 (+ 74us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:19.642679 (+ 60us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:19.642769 (+ 90us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.642933 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":66904,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":37,"raft.queue_time_us":123118,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":1384127,"threads_started":2}
I20260619 21:16:19.595433 3101 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.607051 1657 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38114 (request call id 71) took 1100 ms. Trace:
I20260619 21:16:19.600800 1737 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:19.396505 1805 rpcz_store.cc:276] 0619 21:16:17.729414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.740133 (+ 10719us) service_pool.cc:227] Handling call
0619 21:16:17.754156 (+ 14023us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.757676 (+ 3520us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.126408 (+1368732us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.293672 (+167264us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.294295 (+ 623us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:19.630491 3049 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:19.679512 2900 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:19.434494 3080 raft_consensus.cc:504] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.712376 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 268620us
I20260619 21:16:19.698562 2716 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:19.612708 1727 rpcz_store.cc:276] 0619 21:16:17.987280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.997248 (+ 9968us) service_pool.cc:227] Handling call
0619 21:16:18.033430 (+ 36182us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.040388 (+ 6958us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.181383 (+1140995us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.397996 (+216613us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.602732 (+204736us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":184201,"threads_started":1}
I20260619 21:16:19.405102 1802 rpcz_store.cc:276] 0619 21:16:17.779127 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.844457 (+ 65330us) service_pool.cc:227] Handling call
0619 21:16:17.846587 (+ 2130us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.846687 (+ 100us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.128306 (+1281619us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.295829 (+167523us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.296446 (+ 617us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:19.630232 3108 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:19.727811 2931 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:19.611845 2930 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.613921 3105 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.698913 1664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:19.619481 2734 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:19.734066 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 96518.3us
I20260619 21:16:19.681067 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b6bbbb60204347824250a8d71e6d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:19.515426 3103 raft_consensus.cc:504] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.668175 2684 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.669708 2942 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.673243 2861 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.679347 3028 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.698581 2923 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.694267 3121 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.723891 1808 rpcz_store.cc:276] 0619 21:16:17.714788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.716113 (+ 1325us) service_pool.cc:227] Handling call
0619 21:16:17.968113 (+252000us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.978103 (+ 9990us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.434585 (+1456482us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.664086 (+229501us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.664625 (+ 539us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8783}
@ 0x7f719bd86d50 testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
I20260619 21:16:19.796909 2984 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.769251 1727 tablet_service.cc:1467] Processing CreateTablet for tablet 35cdbd73611d4dcd915a0971b48e3d49 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260619 21:16:20.078259 1860 tablet_bootstrap.cc:492] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:20.052758 2865 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.739279 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 612) took 1118 ms. Trace:
I20260619 21:16:20.078452 1818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 672) took 1378 ms. Trace:
W20260619 21:16:19.842078 1803 consensus_meta.cc:287] T 7f05cdc19f4e420880bab15773a59709 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.710s user 0.009s sys 0.003s
I20260619 21:16:20.021859 3120 raft_consensus.cc:2726] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260619 21:16:20.096436 1818 rpcz_store.cc:276] 0619 21:16:18.699828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.699953 (+ 125us) service_pool.cc:227] Handling call
0619 21:16:18.730821 (+ 30868us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:19.006851 (+276030us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:20.078160 (+1071309us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400134a54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:20.078386 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":623246720}
I20260619 21:16:20.097064 1666 raft_consensus.cc:1261] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer d53146739e3240239c0a3d5ea90e07fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260619 21:16:20.073613 3118 log.cc:924] Time spent T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.332s user 0.001s sys 0.000s
W20260619 21:16:19.745771 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 145054us
I20260619 21:16:20.098529 1803 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 78) took 2343 ms. Trace:
I20260619 21:16:19.850515 3102 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.095835 1860 ts_tablet_manager.cc:1397] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 2.567s user 0.023s sys 0.029s
I20260619 21:16:20.099139 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 670) took 1399 ms. Trace:
I20260619 21:16:19.730132 3101 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.095093 1708 leader_election.cc:302] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:20.099620 1817 rpcz_store.cc:276] 0619 21:16:18.699236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.711609 (+ 12373us) service_pool.cc:227] Handling call
0619 21:16:18.742658 (+ 31049us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:18.802860 (+ 60202us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:18.802898 (+ 38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0619 21:16:18.807344 (+ 4446us) log.cc:842] Serialized 26 byte log entry
0619 21:16:19.147205 (+339861us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b18003305e0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e745411 00007f719ea925cf
0619 21:16:19.427497 (+280292us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:19.498125 (+ 70628us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.498376 (+ 251us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:19.498458 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:20.098813 (+600355us) raft_consensus.cc:1671] finished
0619 21:16:20.098942 (+ 129us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:20.099079 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":115834,"prepare.queue_time_us":87230,"raft.queue_time_us":76829,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":42685,"spinlock_wait_cycles":9979648,"thread_start_us":250405,"threads_started":1,"wal-append.queue_time_us":824027}
I20260619 21:16:20.101298 3101 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.094172 1635 leader_election.cc:302] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:20.098956 1803 rpcz_store.cc:276] 0619 21:16:17.754954 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.825851 (+ 70897us) service_pool.cc:227] Handling call
0619 21:16:17.855961 (+ 30110us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.856123 (+ 162us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.131516 (+1275393us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:20.096811 (+965295us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:20.098439 (+ 1628us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:20.102403 1635 leader_election.cc:302] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:20.103201 2861 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.103659 3120 raft_consensus.cc:2781] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260619 21:16:20.098431 1783 leader_election.cc:302] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:20.109148 3120 raft_consensus.cc:686] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [term 4 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:20.109618 2684 leader_election.cc:288] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.110092 3120 consensus_queue.cc:237] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.111387 2684 consensus_queue.cc:1035] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260619 21:16:19.790504 3116 log.cc:924] Time spent T a192c7be8b2c49bda61f7d5588b3d246 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.276s user 0.001s sys 0.000s
I20260619 21:16:19.784821 2925 raft_consensus.cc:2781] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:20.113355 2925 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:19.986268 3123 tablet_bootstrap.cc:492] T 21131e98f1f94e8cac1998e6c32aa089 P d53146739e3240239c0a3d5ea90e07fe: Bootstrap starting.
I20260619 21:16:20.113694 2925 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:20.105557 1740 raft_consensus.cc:1261] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260619 21:16:20.099123 3103 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.789531 3076 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.114801 3076 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.117404 3076 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:19.727722 2977 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.106542 3105 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.104178 1860 raft_consensus.cc:348] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } }
I20260619 21:16:20.095109 2984 raft_consensus.cc:482] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.119464 1860 raft_consensus.cc:374] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:19.793012 2693 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:20.119993 1860 raft_consensus.cc:729] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:20.119796 2984 raft_consensus.cc:504] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.940315 2980 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.985809 3126 raft_consensus.cc:482] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.121001 1860 consensus_queue.cc:260] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } }
I20260619 21:16:20.122123 2984 leader_election.cc:288] T 84c655fb144a4c11b8902e23b4fac063 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.734428 3078 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.122218 3105 consensus_queue.cc:1046] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:16:20.122812 3078 raft_consensus.cc:504] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.123395 3105 raft_consensus.cc:2781] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader election won for term 6
I20260619 21:16:20.123924 2930 leader_election.cc:288] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.105516 1635 leader_election.cc:302] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:19.789674 3107 raft_consensus.cc:3032] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Stepping down as leader of term 1
I20260619 21:16:20.109525 3117 raft_consensus.cc:2781] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:20.106644 3102 raft_consensus.cc:2726] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260619 21:16:20.124884 3117 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:19.838798 3119 raft_consensus.cc:2781] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260619 21:16:20.125170 3076 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.125362 3117 raft_consensus.cc:3037] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:19.721926 1801 rpcz_store.cc:276] 0619 21:16:17.869492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.871622 (+ 2130us) service_pool.cc:227] Handling call
0619 21:16:18.010233 (+138611us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.015129 (+ 4896us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.191322 (+1176193us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.420410 (+229088us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.433474 (+ 13064us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:19.769186 1799 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 81) took 1892 ms. Trace:
I20260619 21:16:20.126889 3076 raft_consensus.cc:504] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.127074 1799 rpcz_store.cc:276] 0619 21:16:17.876351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.935838 (+ 59487us) service_pool.cc:227] Handling call
0619 21:16:18.480959 (+545121us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a6cd21 00007f71a3a4a1e3 00007f71a39fb869 00007f719cffdc44 00007f719cffdbd2 00007f719cffdb01
0619 21:16:18.481144 (+ 185us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.481264 (+ 120us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:18.919508 (+438244us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.673798 (+754290us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.751879 (+ 78081us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":25138048}
I20260619 21:16:20.129721 2734 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.040577 3048 raft_consensus.cc:482] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.130494 3048 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.132473 3048 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:19.750438 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f05cdc19f4e420880bab15773a59709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.133071 3076 leader_election.cc:288] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.134491 2923 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:19.725123 1657 rpcz_store.cc:276] 0619 21:16:18.418008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.418512 (+ 504us) service_pool.cc:227] Handling call
0619 21:16:18.420557 (+ 2045us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.420663 (+ 106us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.000998 (+580335us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.415405 (+414407us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:19.518076 (+102671us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12035}
I20260619 21:16:20.135021 2923 raft_consensus.cc:482] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.135778 2923 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.136484 1657 tablet_service.cc:1467] Processing CreateTablet for tablet 35cdbd73611d4dcd915a0971b48e3d49 (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260619 21:16:20.137032 3107 raft_consensus.cc:729] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:20.121830 3126 raft_consensus.cc:504] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:19.839160 3129 consensus_queue.cc:1035] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.268s
I20260619 21:16:20.120548 2693 raft_consensus.cc:504] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:20.189334 1784 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.550s user 0.002s sys 0.009s
I20260619 21:16:20.226305 1741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.144658 1667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 607) took 1906 ms. Trace:
I20260619 21:16:20.095868 1661 rpcz_store.cc:276] 0619 21:16:18.620927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.621086 (+ 159us) service_pool.cc:227] Handling call
0619 21:16:18.664490 (+ 43404us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:18.664685 (+ 195us) raft_consensus.cc:1520] Early marking committed up to index 3
0619 21:16:19.611644 (+946959us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.738889 (+127245us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:19.738967 (+ 78us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:19.739011 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:19.739083 (+ 72us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:19.739216 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":340503,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":102511,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":1020502,"threads_started":2}
I20260619 21:16:20.135195 3076 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:20.249519 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 154488us
W20260619 21:16:20.242131 1784 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.603s user 0.004s sys 0.009s
W20260619 21:16:19.908095 3091 log.cc:924] Time spent T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260619 21:16:20.253684 1708 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
W20260619 21:16:20.258222 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 152762us
I20260619 21:16:20.137876 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cdbd73611d4dcd915a0971b48e3d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.243011 1667 rpcz_store.cc:276] 0619 21:16:18.237840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.237966 (+ 126us) service_pool.cc:227] Handling call
0619 21:16:18.241312 (+ 3346us) raft_consensus.cc:1457] Updating replica for 2 ops
0619 21:16:18.244156 (+ 2844us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:18.244188 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0619 21:16:18.244819 (+ 631us) log.cc:842] Serialized 52 byte log entry
0619 21:16:18.711474 (+466655us) raft_consensus.cc:1631] Marking committed up to 4
0619 21:16:19.636993 (+925519us) log.cc:842] Serialized 14 byte log entry
0619 21:16:19.668333 (+ 31340us) log.cc:842] Serialized 14 byte log entry
0619 21:16:20.144179 (+475846us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:20.144292 (+ 113us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0619 21:16:20.144357 (+ 65us) raft_consensus.cc:1671] finished
0619 21:16:20.144375 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:20.144574 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":32909,"prepare.queue_time_us":358263,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":275800,"raft.run_cpu_time_us":445,"raft.run_wall_time_us":47320,"thread_start_us":1812173,"threads_started":3,"wal-append.queue_time_us":628583}
W20260619 21:16:20.244534 2981 consensus_meta.cc:287] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 2.023s user 0.027s sys 0.020s
W20260619 21:16:20.249982 3029 consensus_meta.cc:287] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.846s user 0.013s sys 0.013s
I20260619 21:16:19.847417 3124 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.137705 2923 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.124874 3078 leader_election.cc:288] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.096055 3121 leader_election.cc:288] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:20.237291 3083 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s user 0.000s sys 0.021s
I20260619 21:16:20.123915 3105 raft_consensus.cc:686] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [term 6 LEADER]: Becoming Leader. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:20.213514 2930 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.119747 2977 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.238132 3128 raft_consensus.cc:482] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:19.796732 2809 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.141101 3080 leader_election.cc:288] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.243153 3129 consensus_queue.cc:1225] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer b2e7f02778ec43bdbca03dd013687c76 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260619 21:16:19.849797 3074 leader_election.cc:288] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.257445 2806 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:20.265408 3119 raft_consensus.cc:686] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [term 5 LEADER]: Becoming Leader. State: Replica: 884c214bb0a7455d91701ec388635e74, State: Running, Role: LEADER
I20260619 21:16:19.739578 3049 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:20.290330 3046 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.521s user 0.006s sys 0.013s
I20260619 21:16:20.274797 3102 raft_consensus.cc:2726] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260619 21:16:20.290696 2851 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.288527 2734 raft_consensus.cc:482] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.362322 1815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9a51e66e0646eeb7161c0cb670ae77" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:20.296283 3083 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s user 0.000s sys 0.022s
W20260619 21:16:20.335632 1615 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260619 21:16:20.265765 3076 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.377338 1736 raft_consensus.cc:3032] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:20.410604 1736 raft_consensus.cc:729] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:20.377146 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.378474 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6984e19be5654fef830977bff30f643e" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.302572 3029 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.379545 1860 ts_tablet_manager.cc:1428] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.280s user 0.026s sys 0.027s
I20260619 21:16:20.296268 2672 raft_consensus.cc:482] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.423388 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.384449 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.386384 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:20.401432 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f05cdc19f4e420880bab15773a59709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.287598 2981 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.350222 2923 raft_consensus.cc:482] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.412125 1811 raft_consensus.cc:2454] T 6984e19be5654fef830977bff30f643e P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
W20260619 21:16:20.415679 1710 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.685s user 0.093s sys 0.089s
W20260619 21:16:20.388298 2716 consensus_meta.cc:287] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.574s user 0.008s sys 0.002s
I20260619 21:16:20.404837 1815 raft_consensus.cc:2396] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260619 21:16:20.447295 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.378933 3121 raft_consensus.cc:482] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.251345 3108 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:20.409345 2975 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s user 0.023s sys 0.057s
I20260619 21:16:20.138150 3107 consensus_queue.cc:260] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.371825 3143 raft_consensus.cc:2726] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260619 21:16:20.395810 3103 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.396535 3074 raft_consensus.cc:482] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.354960 2931 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.354521 2930 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.424670 1736 consensus_queue.cc:260] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.448210 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.745255 1738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 713) took 2086 ms. Trace:
I20260619 21:16:20.746480 1738 rpcz_store.cc:276] 0619 21:16:18.659107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.739282 (+ 80175us) service_pool.cc:227] Handling call
0619 21:16:19.193045 (+453763us) raft_consensus.cc:1457] Updating replica for 1 ops
0619 21:16:19.193923 (+ 878us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:20.744902 (+1550979us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400093b50. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabda09 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:20.745179 (+ 277us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":887686912}
W20260619 21:16:20.735540 1710 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.011s user 0.113s sys 0.101s
W20260619 21:16:20.748548 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.05263e+06us
I20260619 21:16:20.450099 1809 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 2.
I20260619 21:16:20.743152 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:20.698559 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 439551us
I20260619 21:16:20.508121 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.591320 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:20.448640 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655f9756924a470b8654421a17c9322e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.749470 1710 leader_election.cc:302] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:20.635303 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 384787us
I20260619 21:16:20.754909 1736 raft_consensus.cc:3037] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
W20260619 21:16:20.618136 1725 consensus_meta.cc:287] T 655f9756924a470b8654421a17c9322e P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.488s user 0.017s sys 0.003s
W20260619 21:16:20.749861 1782 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.125s user 0.000s sys 0.010s
I20260619 21:16:20.447934 1739 raft_consensus.cc:2454] T cfa69d158f2b471e8638824d3d9c757b P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
W20260619 21:16:20.760828 1782 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.136s user 0.000s sys 0.011s
I20260619 21:16:20.532493 1814 raft_consensus.cc:1261] T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:20.750975 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.384112 3131 raft_consensus.cc:482] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.363574 3128 raft_consensus.cc:504] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.755523 1660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 598) took 2689 ms. Trace:
I20260619 21:16:20.811844 1660 rpcz_store.cc:276] 0619 21:16:18.057022 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.065085 (+ 8063us) service_pool.cc:227] Handling call
0619 21:16:18.065477 (+ 392us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:19.599917 (+1534440us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:20.746074 (+1146157us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003b4054. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:20.746295 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3143195392}
I20260619 21:16:20.752779 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.732034 1860 tablet_bootstrap.cc:492] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
W20260619 21:16:20.744777 2975 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s user 0.029s sys 0.060s
W20260619 21:16:20.504339 3127 log.cc:924] Time spent T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.206s user 0.000s sys 0.000s
W20260619 21:16:20.645720 3137 log.cc:924] Time spent T 00000000000000000000000000000000 P 2027366fd3a2418280b4f598c8498021: Append to log took a long time: real 0.568s user 0.000s sys 0.002s
I20260619 21:16:20.758777 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.406533 2851 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.587342 3120 raft_consensus.cc:482] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.745522 1813 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 6.
I20260619 21:16:20.810248 1669 raft_consensus.cc:1261] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260619 21:16:20.761761 1665 raft_consensus.cc:2454] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:20.785993 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.823108 1661 raft_consensus.cc:2454] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:20.813619 1668 raft_consensus.cc:2454] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:20.824361 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.750574 3040 raft_consensus.cc:482] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.814589 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.831554 1739 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:20.840921 1666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38122 (request call id 644) took 1443 ms. Trace:
I20260619 21:16:20.632856 3129 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.503494 3145 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:20.648406 3046 raft_consensus.cc:482] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.616221 3133 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:20.638727 3138 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:20.872327 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.872614 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.815150 3101 raft_consensus.cc:482] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.639223 3139 raft_consensus.cc:482] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.616463 3028 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.617841 3076 leader_election.cc:288] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.825183 1810 raft_consensus.cc:2379] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 4: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:20.461329 2806 raft_consensus.cc:482] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.754467 2861 consensus_queue.cc:1046] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.116s
I20260619 21:16:20.827507 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c1f262670941baa9933b28f3c67fc2" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.755247 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.817873 3143 raft_consensus.cc:482] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.416832 2734 raft_consensus.cc:504] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.837522 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.841384 1666 rpcz_store.cc:276] 0619 21:16:19.397365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.398427 (+ 1062us) service_pool.cc:227] Handling call
0619 21:16:19.434229 (+ 35802us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:20.097677 (+663448us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:20.840677 (+743000us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54000aa354. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:20.840844 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1523304320}
I20260619 21:16:20.475674 3102 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:20.875162 1743 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:20.875993 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.447645 3029 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.586782 3117 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:20.837217 1554 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 341810us
I20260619 21:16:20.826193 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.847240 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18756ff2e1e84eb5b7139ecabf689d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.757416 2923 raft_consensus.cc:504] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.542233 3105 consensus_queue.cc:237] T bfe558fbf8e24ddfbe844d69487cce1d P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.293840 3124 raft_consensus.cc:504] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.461537 2672 raft_consensus.cc:504] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:20.818748 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 69390us
I20260619 21:16:20.890051 1661 raft_consensus.cc:1261] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260619 21:16:20.896479 1665 raft_consensus.cc:2454] T 13c1f262670941baa9933b28f3c67fc2 P 884c214bb0a7455d91701ec388635e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 7.
I20260619 21:16:20.817737 3148 raft_consensus.cc:482] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.757318 3074 raft_consensus.cc:504] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.750905 3103 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.752866 2716 raft_consensus.cc:504] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.896749 1662 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:20.902734 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.838181 2987 leader_election.cc:288] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:20.830094 2942 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.014462 2942 raft_consensus.cc:2781] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:21.014824 2942 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.015146 2942 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:21.016100 1800 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 76) took 3299 ms. Trace:
I20260619 21:16:20.832388 2975 raft_consensus.cc:482] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.829420 2684 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.841658 3150 raft_consensus.cc:482] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.954836 1660 raft_consensus.cc:2454] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:20.870183 3040 raft_consensus.cc:504] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } }
I20260619 21:16:20.638631 3119 consensus_queue.cc:237] T deaff01bc5d54f808990dbcbaf46c0b1 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.888489 3138 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.843381 2977 raft_consensus.cc:482] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:21.043501 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:20.960444 1814 raft_consensus.cc:2354] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 6. Current term is 7.
I20260619 21:16:20.971598 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:20.973887 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 132903us
I20260619 21:16:20.871035 3133 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:20.887215 1651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7a69751ebea42d5950577710f9b1301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.843866 3144 raft_consensus.cc:482] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:20.829260 3142 raft_consensus.cc:482] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:20.752285 3121 raft_consensus.cc:504] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.054481 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.997570 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.997690 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.997809 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:20.957535 1807 consensus_meta.cc:287] T 96effdfbfd2640ce8ee752e99d252d9e P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.769s user 0.013s sys 0.000s
I20260619 21:16:20.999648 1818 raft_consensus.cc:1261] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260619 21:16:21.000334 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa69d158f2b471e8638824d3d9c757b" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:21.003018 1816 raft_consensus.cc:1226] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Rejecting Update request from peer 884c214bb0a7455d91701ec388635e74 for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:21.003921 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.005204 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.831944 3120 raft_consensus.cc:504] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.013366 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eaefe3f12e418dbd2cdd20c2a3e228" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.972577 3102 raft_consensus.cc:504] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:21.008432 1654 thread.cc:636] tablet-open [worker] (thread pool) Time spent creating pthread: real 1.591s user 0.025s sys 0.019s
I20260619 21:16:21.017238 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:21.018455 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.021073 3147 consensus_queue.cc:1035] T 8e9a51e66e0646eeb7161c0cb670ae77 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.180s
W20260619 21:16:21.021687 1663 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38122 (request call id 649) took 1430 ms (client timeout 1501 ms). Trace:
W20260619 21:16:21.066093 1663 rpcz_store.cc:269] 0619 21:16:19.591815 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.602183 (+ 10368us) service_pool.cc:227] Handling call
0619 21:16:21.021499 (+1419316us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b54002e3454. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:21.021627 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":44018432}
I20260619 21:16:20.968597 2861 consensus_queue.cc:1225] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Peer 884c214bb0a7455d91701ec388635e74 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260619 21:16:21.024581 1741 raft_consensus.cc:2454] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
W20260619 21:16:20.990850 2775 consensus_meta.cc:287] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.251s user 0.013s sys 0.000s
I20260619 21:16:20.957154 1725 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35686 (request call id 77) took 2943 ms. Trace:
I20260619 21:16:20.822791 3131 raft_consensus.cc:504] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.037295 1860 tablet_bootstrap.cc:654] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:16:20.998494 1652 consensus_meta.cc:287] T 84b6bbbb60204347824250a8d71e6d3c P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.574s user 0.015s sys 0.004s
W20260619 21:16:21.004860 2979 consensus_meta.cc:287] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.296s user 0.013s sys 0.001s
I20260619 21:16:20.975113 3028 raft_consensus.cc:482] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:21.039914 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 254586us
W20260619 21:16:21.065761 1654 thread.cc:602] tablet-open [worker] (thread pool) Time spent starting thread: real 1.650s user 0.026s sys 0.019s
I20260619 21:16:21.102080 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.896471 3101 raft_consensus.cc:504] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.045475 1743 raft_consensus.cc:2379] T 34c673e979964335b28709efdae7ece1 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:21.046275 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.114493 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.115964 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:20.899339 3046 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.049335 1735 raft_consensus.cc:1226] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: []
I20260619 21:16:20.924685 2806 raft_consensus.cc:504] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:21.031615 2900 consensus_meta.cc:287] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.248s user 0.013s sys 0.000s
I20260619 21:16:21.057544 1810 raft_consensus.cc:2454] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:21.009795 2930 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.016477 1800 rpcz_store.cc:276] 0619 21:16:17.716865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.730524 (+ 13659us) service_pool.cc:227] Handling call
0619 21:16:17.741938 (+ 11414us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.744638 (+ 2700us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:20.541930 (+2797292us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.015571 (+473641us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:21.016034 (+ 463us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":19328}
I20260619 21:16:21.125243 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.126149 1810 raft_consensus.cc:2396] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:20.883677 3145 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.056707 1809 raft_consensus.cc:2454] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:21.056452 1813 raft_consensus.cc:2454] T b1857ce01be64d68891414a298ac40af P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:21.066144 2981 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.061975 1662 raft_consensus.cc:2379] T 68eaefe3f12e418dbd2cdd20c2a3e228 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:21.116694 1814 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 3.
I20260619 21:16:21.128533 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c673e979964335b28709efdae7ece1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:21.129174 1662 raft_consensus.cc:2454] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
I20260619 21:16:20.983127 3143 raft_consensus.cc:504] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.130249 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:21.131372 1662 raft_consensus.cc:2396] T 225b6f9e181a43de8dceeb75cbb1c01d P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:21.131556 3143 leader_election.cc:288] T ffcc16880ccb4a85a786728439b64cc0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.125018 1654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38114 (request call id 72) took 2706 ms. Trace:
I20260619 21:16:21.063004 1661 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:21.071300 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:21.133627 1665 raft_consensus.cc:2454] T 2aedd754fcc34eecbb29d5cccbb9153a P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:21.075486 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:21.078634 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.041531 1727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cdbd73611d4dcd915a0971b48e3d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:20.896744 3139 raft_consensus.cc:504] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.107998 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:21.138033 1811 raft_consensus.cc:2379] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 2: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:21.060571 3103 leader_election.cc:288] T 246c3d015ec242eca8bfaf5a076692b1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.112188 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84119be03678450983c7d5a686bd3c86" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:20.903494 3129 raft_consensus.cc:504] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.065583 2984 raft_consensus.cc:482] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.169401 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.042879 2684 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.062031 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:21.138175 1666 raft_consensus.cc:2454] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 2.
I20260619 21:16:21.132473 1654 rpcz_store.cc:276] 0619 21:16:18.418332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.418504 (+ 172us) service_pool.cc:227] Handling call
0619 21:16:18.421858 (+ 3354us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.421976 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.134662 (+712686us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:19.253826 (+119164us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:20.890150 (+1636324us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b44000232cc. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719baa8a95 00007f719baa94b2 00007f719e751511 00007f719e74ce1a 00007f719bb785fe 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f71a3a4a672
0619 21:16:21.109086 (+218936us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:16:21.124888 (+ 15802us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":26180352,"tablet-open.queue_time_us":1740884,"thread_start_us":1685080,"threads_started":1}
I20260619 21:16:21.077153 2931 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.052135 2977 raft_consensus.cc:504] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:21.139555 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 72862.6us
I20260619 21:16:21.038295 2975 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.044215 3150 raft_consensus.cc:504] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.073843 2775 raft_consensus.cc:504] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.168529 1660 raft_consensus.cc:2454] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 3.
W20260619 21:16:21.148412 3146 log.cc:924] Time spent T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.172s user 0.000s sys 0.001s
I20260619 21:16:21.081477 2979 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.149458 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:21.245461 1743 raft_consensus.cc:2396] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260619 21:16:21.197013 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 96effdfbfd2640ce8ee752e99d252d9e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260619 21:16:21.136092 2900 raft_consensus.cc:504] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.197588 3129 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.147671 3131 leader_election.cc:288] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.187781 1739 raft_consensus.cc:2396] T dfef072e2311405996f05452211cddca P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260619 21:16:21.261479 1738 raft_consensus.cc:2379] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:21.165915 1736 raft_consensus.cc:1261] T d3404667de13461aa4b3e79e8d6a64ee P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260619 21:16:21.262636 1736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 724) took 1987 ms. Trace:
I20260619 21:16:21.146275 3102 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:21.202292 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 213251us
I20260619 21:16:21.263089 1736 rpcz_store.cc:276] 0619 21:16:19.275444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.275579 (+ 135us) service_pool.cc:227] Handling call
0619 21:16:19.323351 (+ 47772us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:20.377202 (+1053851us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540023ab20. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719ea4e730 00007f719eac0300 00007f719eac1371 00007f719eabdedf 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc
0619 21:16:21.262338 (+885136us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:21.262569 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":235502464}
I20260619 21:16:21.209357 1782 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.072000 1725 rpcz_store.cc:276] 0619 21:16:18.013662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.037102 (+ 23440us) service_pool.cc:227] Handling call
0619 21:16:18.048927 (+ 11825us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.093652 (+ 44725us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.129849 (+1036197us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:20.822233 (+1692384us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:20.957042 (+134809us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:21.010232 3148 raft_consensus.cc:504] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.141314 3145 leader_election.cc:288] T f0b6191588f14043884eceae0b64c41c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.133543 3108 leader_election.cc:288] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:21.226989 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 145896us
I20260619 21:16:21.060997 3142 raft_consensus.cc:504] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.228399 3029 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.228775 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246c3d015ec242eca8bfaf5a076692b1" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.229771 1652 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38114 (request call id 74) took 2525 ms. Trace:
I20260619 21:16:21.230566 2716 leader_election.cc:288] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.234423 2806 leader_election.cc:288] T a3c98fc9dacf48e8994724fe2e303166 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.234546 2684 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.235620 2734 leader_election.cc:288] T 0fe8c95af50740a783386324c1649be0 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.237962 2975 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:21.134189 3152 log.cc:924] Time spent T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260619 21:16:21.289846 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.290511 1814 raft_consensus.cc:2454] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:21.291314 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.202906 1656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016af7231f1345d3b13158794ac82632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:21.281618 3156 tablet_bootstrap.cc:492] T 6ae495cfaaca419c8611fe233762a02a P 884c214bb0a7455d91701ec388635e74: Bootstrap starting.
I20260619 21:16:21.242061 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f709f3b1e904d1ba8bf5592d77cc8d6" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.246471 1666 raft_consensus.cc:1261] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:21.058437 3144 raft_consensus.cc:504] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.118870 3028 raft_consensus.cc:504] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.305063 2979 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.287635 2923 leader_election.cc:288] T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.306519 3028 leader_election.cc:288] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.259644 1737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:21.307335 1817 raft_consensus.cc:2454] T d80394dabe604f8898e7b24149fc624f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:21.308887 2977 leader_election.cc:288] T c28152c520fe41e5be2905e6e3b67b8a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:21.310473 1806 consensus_meta.cc:287] T 18756ff2e1e84eb5b7139ecabf689d6a P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.527s user 0.011s sys 0.000s
I20260619 21:16:21.311519 1806 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 73) took 3828 ms. Trace:
I20260619 21:16:21.311842 1806 rpcz_store.cc:276] 0619 21:16:17.483439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.485800 (+ 2361us) service_pool.cc:227] Handling call
0619 21:16:17.706642 (+220842us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.714618 (+ 7976us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.698101 (+1983483us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.310971 (+1612870us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:21.311459 (+ 488us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:21.206826 1807 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44456 (request call id 82) took 3295 ms. Trace:
I20260619 21:16:21.312633 2693 leader_election.cc:288] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.313020 1807 rpcz_store.cc:276] 0619 21:16:17.907102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.957385 (+ 50283us) service_pool.cc:227] Handling call
0619 21:16:18.342487 (+385102us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5ee82 00007f71a4a778b9 00007f71a398a5d8 00007f71a3a4a2a0 00007f71a39fb869 00007f719cffdc44 00007f719cffdbd2 00007f719cffdb01 00007f719cffda7c
0619 21:16:18.342573 (+ 86us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.342699 (+ 126us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:20.065745 (+1723046us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.077245 (+1011500us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:21.202837 (+125592us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3867520}
I20260619 21:16:21.313750 3120 leader_election.cc:288] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.313822 1725 tablet_service.cc:1467] Processing CreateTablet for tablet 96effdfbfd2640ce8ee752e99d252d9e (DEFAULT_TABLE table=test-workload [id=1995c89d987e45a6af7b17d850db3dbb]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260619 21:16:21.314379 1740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35690 (request call id 622) took 1957 ms. Trace:
W20260619 21:16:21.260990 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 189856us
I20260619 21:16:21.314780 1740 rpcz_store.cc:276] 0619 21:16:19.357167 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.604438 (+247271us) service_pool.cc:227] Handling call
0619 21:16:19.604751 (+ 313us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:20.114131 (+509380us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:21.314142 (+1200011us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54002eefd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:21.314317 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1150835072}
I20260619 21:16:21.315829 3142 leader_election.cc:288] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.316452 1783 leader_election.cc:302] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:21.317608 1783 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:21.314882 3040 leader_election.cc:288] T 21131e98f1f94e8cac1998e6c32aa089 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637)
I20260619 21:16:21.318940 3133 leader_election.cc:288] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.319417 1783 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
W20260619 21:16:21.270639 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 61358.6us
I20260619 21:16:21.276129 1743 raft_consensus.cc:1261] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Refusing update from remote peer b2e7f02778ec43bdbca03dd013687c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260619 21:16:21.319851 3080 consensus_queue.cc:1046] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 21:16:21.272815 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.314847 1741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 732) took 1165 ms. Trace:
I20260619 21:16:21.283524 1860 tablet_bootstrap.cc:492] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: No bootstrap required, opened a new log
I20260619 21:16:21.282079 3154 raft_consensus.cc:2781] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.202514 2984 raft_consensus.cc:504] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.326161 3029 raft_consensus.cc:2781] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:21.321616 1741 rpcz_store.cc:276] 0619 21:16:20.149583 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:20.149727 (+ 144us) service_pool.cc:227] Handling call
0619 21:16:21.314683 (+1164956us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400012cd4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:21.314789 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1797971200}
W20260619 21:16:21.285904 3114 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260619 21:16:21.262169 2920 raft_consensus.cc:482] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:21.287379 1809 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 6.
I20260619 21:16:21.255832 1635 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:16:21.328436 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 72679.9us
I20260619 21:16:21.329788 1635 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.297457 1818 raft_consensus.cc:2454] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 1.
I20260619 21:16:21.300302 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe8c95af50740a783386324c1649be0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:21.302341 2925 consensus_meta.cc:287] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.188s user 0.014s sys 0.000s
I20260619 21:16:21.303360 3045 ts_tablet_manager.cc:1918] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260619 21:16:21.284132 2672 leader_election.cc:288] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.202747 2930 raft_consensus.cc:482] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:21.321806 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.322400 1860 ts_tablet_manager.cc:1397] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: Time spent bootstrapping tablet: real 0.591s user 0.013s sys 0.025s
I20260619 21:16:21.250308 3128 leader_election.cc:288] T 4d7929378ea14d5fbd2b7c633537d332 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.333168 1663 raft_consensus.cc:2396] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260619 21:16:21.287217 3074 leader_election.cc:288] T f756a9c8b10a4992950255915336bb2c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.144788 3158 consensus_queue.cc:1046] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.155s
I20260619 21:16:21.284502 1652 rpcz_store.cc:276] 0619 21:16:18.704280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.738019 (+ 33739us) service_pool.cc:227] Handling call
0619 21:16:18.832464 (+ 94445us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.832582 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:19.423767 (+591185us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.126948 (+1703181us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:21.229688 (+102740us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:21.344723 2925 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.345268 3040 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260619 21:16:21.335042 1708 leader_election.cc:302] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters: b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:21.358244 1782 leader_election.cc:302] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.268342 3164 raft_consensus.cc:482] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:21.345739 2920 raft_consensus.cc:504] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.326824 3029 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.320029 3133 raft_consensus.cc:2781] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.314802 3046 leader_election.cc:288] T ef644e16cf5b46b68fde33b180432159 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:21.455973 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 97438us
I20260619 21:16:21.468792 1635 leader_election.cc:302] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.469147 1727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35686 (request call id 81) took 3232 ms. Trace:
I20260619 21:16:21.469491 1727 rpcz_store.cc:276] 0619 21:16:18.610703 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.768140 (+1157437us) service_pool.cc:227] Handling call
0619 21:16:20.095434 (+327294us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:20.095527 (+ 93us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:21.254641 (+1159114us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.328479 (+ 73838us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:21.469091 (+140612us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260619 21:16:21.423336 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 59044us
W20260619 21:16:21.459695 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 101482us
I20260619 21:16:21.326171 3154 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.347153 3045 log.cc:1196] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Deleting WAL directory at /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-2-root/wals/5ae5f9625019495d85bc13c82d7f621b
I20260619 21:16:21.377220 3145 consensus_queue.cc:1035] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260619 21:16:21.362457 3142 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:21.360270 2980 raft_consensus.cc:2781] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:21.462253 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96effdfbfd2640ce8ee752e99d252d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:21.458447 3143 raft_consensus.cc:482] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:21.474718 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 45829.7us
I20260619 21:16:21.360872 3124 leader_election.cc:288] T 0dea6228e49c44b9889aa709158ef557 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.458624 3040 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.474503 1635 leader_election.cc:398] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 6. Current term is 7.
I20260619 21:16:21.458781 3120 raft_consensus.cc:482] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:21.465432 3029 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:21.363531 3129 consensus_queue.cc:1035] T 13c1f262670941baa9933b28f3c67fc2 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.113s
I20260619 21:16:21.467320 3079 raft_consensus.cc:2781] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260619 21:16:21.486747 1635 leader_election.cc:401] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260619 21:16:21.459059 1554 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 101817us
I20260619 21:16:21.419894 2930 raft_consensus.cc:504] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.466156 3133 raft_consensus.cc:482] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:21.474956 3103 raft_consensus.cc:2781] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260619 21:16:21.496990 3103 raft_consensus.cc:482] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.497514 3103 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:21.480535 2980 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.492875 1782 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.493837 3153 raft_consensus.cc:2781] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.495935 1635 leader_election.cc:302] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.326717 3163 consensus_queue.cc:1046] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.152s
I20260619 21:16:21.475278 3154 raft_consensus.cc:3037] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:21.462281 3128 consensus_queue.cc:1035] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260619 21:16:21.481058 3142 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.560731 1860 raft_consensus.cc:348] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.562187 1860 raft_consensus.cc:374] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 21:16:21.468502 2693 consensus_queue.cc:1035] T fc672e451a9f4f6eb43430477e3acd1b P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260619 21:16:21.562606 1860 raft_consensus.cc:729] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Initialized, Role: FOLLOWER
I20260619 21:16:21.468357 3164 raft_consensus.cc:504] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.482127 3101 leader_election.cc:288] T 692d38d1f19c468eae7077dde218a62f P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.482342 3121 leader_election.cc:288] T c8a530b7f9264d519f65caf431a0a478 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.489538 3040 raft_consensus.cc:3037] T 044f6297187147aba0e5ec306cc3840a P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:21.539855 2979 raft_consensus.cc:482] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.503827 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96effdfbfd2640ce8ee752e99d252d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 21:16:21.494607 3079 raft_consensus.cc:482] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.484714 3143 raft_consensus.cc:504] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.502732 2775 leader_election.cc:288] T fc672e451a9f4f6eb43430477e3acd1b P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:21.641249 1635 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 145319us
I20260619 21:16:21.502029 2734 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 7
I20260619 21:16:21.612751 1783 leader_election.cc:302] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:21.645372 2977 raft_consensus.cc:2781] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.671152 2925 leader_election.cc:288] T 5ce345f630e142ac9300688d473743af P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.564532 2984 leader_election.cc:288] T dfef072e2311405996f05452211cddca P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.623173 2980 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:21.680119 1635 leader_election.cc:302] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.499794 3133 raft_consensus.cc:3037] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:21.565507 3142 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:21.582383 3045 ts_tablet_manager.cc:1939] T 5ae5f9625019495d85bc13c82d7f621b P b2e7f02778ec43bdbca03dd013687c76: Deleting consensus metadata
I20260619 21:16:21.671989 3121 raft_consensus.cc:2781] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260619 21:16:21.696048 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 83401.5us
I20260619 21:16:21.698768 1635 leader_election.cc:302] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.592475 3138 leader_election.cc:288] T 516fa114155140f6b508f311c4971c08 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.671989 3079 raft_consensus.cc:3037] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Advancing to term 8
I20260619 21:16:21.642645 3139 leader_election.cc:288] T ccfc3736d63b4f80ab5f68b26ab104a0 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.612066 2900 leader_election.cc:288] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 election: Requested vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.696602 1860 consensus_queue.cc:260] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.702210 1635 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
W20260619 21:16:21.696992 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 204093us
I20260619 21:16:21.562817 3148 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:21.678982 2977 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.597828 3153 raft_consensus.cc:482] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:21.632719 3078 consensus_queue.cc:1035] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260619 21:16:21.624317 2979 raft_consensus.cc:504] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.494781 3120 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.721479 1782 leader_election.cc:302] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.701275 2984 raft_consensus.cc:2781] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.699033 3121 raft_consensus.cc:482] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.718089 3150 leader_election.cc:288] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:21.775512 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 54095.9us
W20260619 21:16:21.759198 1659 consensus_meta.cc:287] T 7f05cdc19f4e420880bab15773a59709 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 0.969s user 0.014s sys 0.005s
I20260619 21:16:21.713213 3101 raft_consensus.cc:2781] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260619 21:16:21.754930 2977 raft_consensus.cc:3037] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:21.779362 1635 leader_election.cc:302] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.702205 3074 raft_consensus.cc:2781] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.776463 1782 leader_election.cc:302] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:21.713531 1554 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 151642us
I20260619 21:16:21.716435 3171 raft_consensus.cc:3032] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:21.713443 2920 leader_election.cc:288] T 25c06c6619d04c58a5ed3f65b1c82f70 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.781263 1782 leader_election.cc:302] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e7f02778ec43bdbca03dd013687c76; no voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.757359 2900 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:21.761211 3153 raft_consensus.cc:3037] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:21.792222 2900 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.790755 1635 leader_election.cc:302] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters:
I20260619 21:16:21.791741 1782 leader_election.cc:302] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.802608 1782 leader_election.cc:302] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76; no voters: d53146739e3240239c0a3d5ea90e07fe
I20260619 21:16:21.778528 3028 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:21.778976 2984 raft_consensus.cc:482] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.779771 3101 raft_consensus.cc:482] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.778038 3121 raft_consensus.cc:3037] T bb9b500bf9a24148b2fbcd762b549521 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:21.805600 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 470578us
I20260619 21:16:21.778146 3147 raft_consensus.cc:482] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:21.792793 2900 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:21.811146 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.812150 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.815956 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.773303 3048 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260619 21:16:21.789561 3145 raft_consensus.cc:2781] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:21.783987 3074 raft_consensus.cc:482] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:21.823231 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.792912 3164 leader_election.cc:288] T 32ea84a8ad12422f8ec0ede64598fa44 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.785804 3171 raft_consensus.cc:729] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d53146739e3240239c0a3d5ea90e07fe, State: Running, Role: LEADER
I20260619 21:16:21.826327 1708 leader_election.cc:302] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:21.790181 3046 raft_consensus.cc:2781] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260619 21:16:21.793906 2925 raft_consensus.cc:2726] T 225b6f9e181a43de8dceeb75cbb1c01d P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260619 21:16:21.790879 3045 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44456 (request call id 60) took 4747 ms. Trace:
I20260619 21:16:21.838912 1708 leader_election.cc:302] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:21.808555 2860 raft_consensus.cc:2781] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:21.804216 2684 raft_consensus.cc:2781] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 21:16:21.815059 2984 raft_consensus.cc:3037] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:21.818078 3101 raft_consensus.cc:3037] T b1857ce01be64d68891414a298ac40af P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Advancing to term 6
I20260619 21:16:21.853420 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a530b7f9264d519f65caf431a0a478" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.817956 2693 raft_consensus.cc:482] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:21.854920 1708 leader_election.cc:302] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884c214bb0a7455d91701ec388635e74, d53146739e3240239c0a3d5ea90e07fe; no voters:
I20260619 21:16:21.835316 3074 raft_consensus.cc:3037] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Advancing to term 2
W20260619 21:16:21.925509 1708 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 70728.3us
I20260619 21:16:21.930244 3128 raft_consensus.cc:2781] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 21:16:21.839819 3130 raft_consensus.cc:2781] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260619 21:16:21.930881 3128 raft_consensus.cc:482] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.931254 3130 raft_consensus.cc:482] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.931481 3128 raft_consensus.cc:3037] T 2aedd754fcc34eecbb29d5cccbb9153a P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:21.931762 3130 raft_consensus.cc:3037] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:21.844442 3160 raft_consensus.cc:2726] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:21.939237 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:21.939905 1818 raft_consensus.cc:3037] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:21.928992 1811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.941164 3143 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.941676 1811 raft_consensus.cc:2379] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 6: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:21.848603 2684 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.944886 2684 raft_consensus.cc:3037] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:21.928787 2693 raft_consensus.cc:504] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:21.930212 1860 ts_tablet_manager.cc:1428] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: Time spent starting tablet: real 0.597s user 0.027s sys 0.038s
W20260619 21:16:21.949242 1860 ts_tablet_manager.cc:1460] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: Tablet startup took 616ms
W20260619 21:16:21.949663 1860 ts_tablet_manager.cc:1462] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76: Trace:
0619 21:16:17.490460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.491575 (+ 1115us) service_pool.cc:227] Handling call
0619 21:16:17.496447 (+ 4872us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:17.496551 (+ 104us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:17.792553 (+296002us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:17.853141 (+ 60588us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:17.853626 (+ 485us) inbound_call.cc:173] Queueing success response
0619 21:16:20.731585 (+2877959us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0619 21:16:21.332841 (+601256us) ts_tablet_manager.cc:1429] Starting tablet replica
0619 21:16:21.497428 (+164587us) tablet_replica.cc:259] Starting instrumentation
0619 21:16:21.498111 (+ 683us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":5893,"raft.queue_time_us":7452,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":30245,"tablet-open.queue_time_us":2878054,"thread_start_us":124195,"threads_started":1}
I20260619 21:16:21.835170 3145 raft_consensus.cc:482] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:21.952519 3145 raft_consensus.cc:3037] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:21.953123 3172 consensus_queue.cc:1035] T b1857ce01be64d68891414a298ac40af P d53146739e3240239c0a3d5ea90e07fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:21.845549 2920 raft_consensus.cc:2781] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260619 21:16:21.957823 2920 raft_consensus.cc:482] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 21:16:21.958449 2920 raft_consensus.cc:3037] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:21.840795 3046 raft_consensus.cc:482] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.845218 3144 leader_election.cc:288] T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
W20260619 21:16:21.849072 3170 log.cc:924] Time spent T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 0.306s user 0.000s sys 0.002s
I20260619 21:16:21.782898 3162 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.330s
I20260619 21:16:21.983829 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "516fa114155140f6b508f311c4971c08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:21.856426 1659 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38114 (request call id 78) took 3024 ms. Trace:
I20260619 21:16:21.993515 3158 consensus_queue.cc:1035] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260619 21:16:21.844897 2860 raft_consensus.cc:482] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:21.994854 2860 raft_consensus.cc:3037] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:21.954839 2930 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:21.999374 1816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 661) took 1678 ms. Trace:
I20260619 21:16:21.834411 3147 raft_consensus.cc:504] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.002441 1739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 746) took 1606 ms. Trace:
I20260619 21:16:22.002882 1739 rpcz_store.cc:276] 0619 21:16:20.396237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.262110 (+865873us) service_pool.cc:227] Handling call
0619 21:16:22.001524 (+739414us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b580005fa98. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:22.001793 (+ 269us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:22.002015 (+ 222us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:22.002052 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:22.002119 (+ 67us) raft_consensus.cc:1631] Marking committed up to 2
0619 21:16:22.002160 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.002229 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.002371 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6683904}
I20260619 21:16:21.999802 1816 rpcz_store.cc:276] 0619 21:16:20.320536 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:20.321147 (+ 611us) service_pool.cc:227] Handling call
0619 21:16:20.322852 (+ 1705us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:21.057388 (+734536us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:21.999065 (+941677us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400080554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:21.999292 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1553685376}
I20260619 21:16:21.952818 1860 tablet_bootstrap.cc:492] T 6ae495cfaaca419c8611fe233762a02a P b2e7f02778ec43bdbca03dd013687c76: Bootstrap starting.
I20260619 21:16:21.976011 3046 raft_consensus.cc:3037] T 8133a49f53cd4e0c9bcb3b4c9d5da36d P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
W20260619 21:16:22.065025 1726 consensus_meta.cc:287] T 6ae495cfaaca419c8611fe233762a02a P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.747s user 0.012s sys 0.002s
I20260619 21:16:22.070679 1817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 676) took 1082 ms. Trace:
W20260619 21:16:22.105842 1815 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s user 0.012s sys 0.016s
I20260619 21:16:22.072060 1817 rpcz_store.cc:276] 0619 21:16:20.971051 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:20.971702 (+ 651us) service_pool.cc:227] Handling call
0619 21:16:22.053383 (+1081681us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54001414d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:22.053575 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":574211456}
I20260619 21:16:22.149816 3138 raft_consensus.cc:482] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:22.154675 3176 raft_consensus.cc:482] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:21.991626 1659 rpcz_store.cc:276] 0619 21:16:18.832070 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:18.841106 (+ 9036us) service_pool.cc:227] Handling call
0619 21:16:18.867598 (+ 26492us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.867733 (+ 135us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:20.644890 (+1777157us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.790662 (+1145772us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:21.850761 (+ 60099us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:21.837031 3045 rpcz_store.cc:276] 0619 21:16:17.027903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.046303 (+ 18400us) service_pool.cc:227] Handling call
0619 21:16:17.225163 (+178860us) ts_tablet_manager.cc:1145] Deleting tablet 5ae5f9625019495d85bc13c82d7f621b
0619 21:16:17.233135 (+ 7972us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0619 21:16:20.620780 (+3387645us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.303048 (+682268us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:21.775641 (+472593us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1498127,"tablet-delete.queue_time_us":166006,"thread_start_us":101451,"threads_started":1}
W20260619 21:16:22.233342 1710 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.748s user 0.001s sys 0.014s
I20260619 21:16:22.237907 3176 raft_consensus.cc:504] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.238323 1813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 683) took 1185 ms. Trace:
I20260619 21:16:22.054140 3171 consensus_queue.cc:260] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.194504 3177 raft_consensus.cc:482] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 21:16:22.290144 1782 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.1:45637, user_credentials={real_user=slave}} blocked reactor thread for 487578us
I20260619 21:16:22.292848 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deaff01bc5d54f808990dbcbaf46c0b1" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:22.293871 1710 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.812s user 0.002s sys 0.015s
I20260619 21:16:22.294270 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb9bc6419d14ea8b9834eea6fafc54a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.296696 1813 rpcz_store.cc:276] 0619 21:16:21.052616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.142773 (+ 90157us) service_pool.cc:227] Handling call
0619 21:16:21.205505 (+ 62732us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:21.205714 (+ 209us) raft_consensus.cc:1520] Early marking committed up to index 1
0619 21:16:21.259028 (+ 53314us) log.cc:842] Serialized 14 byte log entry
0619 21:16:21.996753 (+737725us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b180036cca0. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a4a358b1 00007f71a4a42661 00007f719bb78345 00007f71a49afa39 00007f719bb958bb 00007f719bb93cbf 00007f719bb95e22 00007f719e73f5bb 00007f719e745063 00007f719e74557b 00007f719eac888a
0619 21:16:22.237893 (+241140us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:22.237962 (+ 69us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:22.238016 (+ 54us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.238101 (+ 85us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.238254 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":51643,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1304,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":156,"spinlock_wait_cycles":3367168,"thread_start_us":240845,"threads_started":1}
I20260619 21:16:22.237074 2693 leader_election.cc:288] T 9980ccca53c3424db1585f12669d6d58 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:22.239814 3049 leader_election.cc:288] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:22.302935 1666 raft_consensus.cc:2396] T 2cb9bc6419d14ea8b9834eea6fafc54a P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260619 21:16:22.303785 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.314405 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.321163 1668 raft_consensus.cc:2454] T 0d45d883f7fa4dd285976666f28c0770 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:22.321692 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.322135 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:22.304822 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 828846us
I20260619 21:16:22.305114 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.328461 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c98fc9dacf48e8994724fe2e303166" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.328792 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1e21185de74e3da01e18ff99572a08" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.330041 1662 raft_consensus.cc:2354] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5.
I20260619 21:16:22.329908 1743 raft_consensus.cc:2454] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:22.331166 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.322413 1666 raft_consensus.cc:2454] T ffcc16880ccb4a85a786728439b64cc0 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:22.328752 1736 raft_consensus.cc:1261] T cd1e5b0f2736497bb5ea639a3071f8a2 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Refusing update from remote peer 884c214bb0a7455d91701ec388635e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260619 21:16:22.299079 3122 log.cc:924] Time spent T f0b6191588f14043884eceae0b64c41c P 884c214bb0a7455d91701ec388635e74: Append to log took a long time: real 2.641s user 0.004s sys 0.010s
I20260619 21:16:22.237067 3138 raft_consensus.cc:504] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.333398 1668 raft_consensus.cc:2379] T 4d7929378ea14d5fbd2b7c633537d332 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:22.335651 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.337142 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.337504 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.338912 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:22.342895 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.343828 1665 raft_consensus.cc:2454] T 78cb98bb4e2642acb6448e5f5ccdb1ff P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:22.326996 2930 raft_consensus.cc:482] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:22.347637 2930 raft_consensus.cc:504] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.348652 3123 tablet_bootstrap.cc:654] T 21131e98f1f94e8cac1998e6c32aa089 P d53146739e3240239c0a3d5ea90e07fe: Neither blocks nor log segments found. Creating new log.
I20260619 21:16:22.348857 3138 leader_election.cc:288] T da1270002c3c45988c3ac4fbeafaf019 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:22.349371 3171 raft_consensus.cc:3037] T 68eaefe3f12e418dbd2cdd20c2a3e228 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Advancing to term 5
I20260619 21:16:22.351852 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.352636 1816 raft_consensus.cc:2379] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 7: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:22.352891 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.353377 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.295509 2979 leader_election.cc:288] T f9c4f216cdd7450b9659feeea5fd2bf9 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:22.354432 1817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.303277 3136 raft_consensus.cc:482] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 884c214bb0a7455d91701ec388635e74)
I20260619 21:16:22.354813 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.350106 2930 leader_election.cc:288] T 692d38d1f19c468eae7077dde218a62f P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:22.366705 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044f6297187147aba0e5ec306cc3840a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.395808 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
@ 0x7f719bd79e2d testing::UnitTest::Run() at ??:0
I20260619 21:16:22.319182 2931 consensus_queue.cc:1035] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.117s
@ 0x36966c RUN_ALL_TESTS() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:?
@ 0x36852d main at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:?
I20260619 21:16:22.320258 3177 raft_consensus.cc:504] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.313684 1554 catalog_manager.cc:4841] TS b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591): tablet 5ae5f9625019495d85bc13c82d7f621b (table test-workload [id=1995c89d987e45a6af7b17d850db3dbb]) successfully deleted
I20260619 21:16:22.750376 1667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:22.421427 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 473312us
W20260619 21:16:22.474416 1553 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 105390us
I20260619 21:16:22.724793 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6333f19ae36b48459bbfa0490f45d128" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:22.684646 1663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38138 (request call id 645) took 1655 ms. Trace:
I20260619 21:16:22.531208 1661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 647) took 1468 ms. Trace:
I20260619 21:16:22.671245 1735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 726) took 3395 ms. Trace:
I20260619 21:16:22.685464 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 670) took 2053 ms. Trace:
I20260619 21:16:22.528815 1809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 685) took 1457 ms. Trace:
I20260619 21:16:22.314369 1726 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35686 (request call id 72) took 4308 ms. Trace:
I20260619 21:16:22.653154 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.357518 3045 tablet_replica.cc:330] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76: stopping tablet replica
I20260619 21:16:22.754498 1812 rpcz_store.cc:276] 0619 21:16:20.746279 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:20.753617 (+ 7338us) service_pool.cc:227] Handling call
0619 21:16:20.753922 (+ 305us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:21.029708 (+275786us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:21.029741 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:21.029777 (+ 36us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:21.029825 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.685166 (+1655341us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54003ef8d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:22.685223 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.685391 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":633731456}
I20260619 21:16:22.554145 1660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:22.780511 1660 raft_consensus.cc:3037] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:22.752236 1666 raft_consensus.cc:3037] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 4 FOLLOWER]: Advancing to term 5
W20260619 21:16:22.507658 3140 log.cc:924] Time spent T 5e50dfc7ff65482ab6eb4b18b5ae06bb P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 2.016s user 0.000s sys 0.003s
I20260619 21:16:22.752199 1663 rpcz_store.cc:276] 0619 21:16:21.029940 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.067031 (+ 37091us) service_pool.cc:227] Handling call
0619 21:16:22.684319 (+1617288us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400330254. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:22.684545 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":515414272}
I20260619 21:16:22.473531 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 687) took 1389 ms. Trace:
I20260619 21:16:22.755926 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.791616 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.803267 1665 raft_consensus.cc:2379] T efba5ed7ac544242953fd0c25465b088 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e7f02778ec43bdbca03dd013687c76 in current term 5: Already voted for candidate 884c214bb0a7455d91701ec388635e74 in this term.
I20260619 21:16:22.472618 1669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38138 (request call id 626) took 1727 ms. Trace:
I20260619 21:16:22.753980 1735 rpcz_store.cc:276] 0619 21:16:19.276290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.276426 (+ 136us) service_pool.cc:227] Handling call
0619 21:16:19.325422 (+ 48996us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:21.117409 (+1791987us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.670956 (+1553547us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540040ef54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:22.671162 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3963816832}
I20260619 21:16:22.471001 1744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 725) took 3195 ms. Trace:
W20260619 21:16:22.788475 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 1.1382e+06us
I20260619 21:16:22.421536 1740 raft_consensus.cc:2454] T 044f6297187147aba0e5ec306cc3840a P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 6.
I20260619 21:16:22.440619 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efba5ed7ac544242953fd0c25465b088" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.804193 1669 rpcz_store.cc:276] 0619 21:16:20.744711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:20.830465 (+ 85754us) service_pool.cc:227] Handling call
0619 21:16:20.873589 (+ 43124us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:21.024838 (+151249us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:21.024870 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:21.024912 (+ 42us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:21.024971 (+ 59us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.472303 (+1447332us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54000aad54. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:22.472361 (+ 58us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.472544 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":347292032}
I20260619 21:16:22.807952 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.755466 1809 rpcz_store.cc:276] 0619 21:16:21.071554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.127075 (+ 55521us) service_pool.cc:227] Handling call
0619 21:16:22.528613 (+1401538us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540023e1d4. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:22.528742 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":269662080}
W20260619 21:16:22.471868 1738 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 747) took 2055 ms (client timeout 1702 ms). Trace:
W20260619 21:16:22.858942 1738 rpcz_store.cc:269] 0619 21:16:20.416423 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:20.821248 (+404825us) service_pool.cc:227] Handling call
0619 21:16:22.471615 (+1650367us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54001a2554. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eac22ae 00007f71a39ffc27 00007f719d0e7c04 00007f719d0e7b92 00007f719d0e7ac1 00007f719d0e7a3c 00007f719d0e6cf2 00007f719cee894c 00007f719cee7d86 00007f719cee772f
0619 21:16:22.471786 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":675708160}
W20260619 21:16:22.787427 1818 consensus_meta.cc:287] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.847s user 0.014s sys 0.040s
I20260619 21:16:22.796221 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225b6f9e181a43de8dceeb75cbb1c01d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.860248 1739 raft_consensus.cc:2454] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:22.781194 1816 raft_consensus.cc:2454] T 9ae81d060b714ae98341d6b5e2669f3f P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 1.
I20260619 21:16:22.653661 1742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 737) took 2494 ms. Trace:
I20260619 21:16:22.861490 1742 rpcz_store.cc:276] 0619 21:16:20.158744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:22.599694 (+2440950us) service_pool.cc:227] Handling call
0619 21:16:22.652956 (+ 53262us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:22.653181 (+ 225us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:22.653222 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:22.653280 (+ 58us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:22.653320 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.653411 (+ 91us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.653574 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:22.753154 1661 rpcz_store.cc:276] 0619 21:16:21.062715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.213326 (+150611us) service_pool.cc:227] Handling call
0619 21:16:22.530443 (+1317117us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b58002b0598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:22.530666 (+ 223us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:22.530856 (+ 190us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:22.530884 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:22.530931 (+ 47us) raft_consensus.cc:1631] Marking committed up to 1
0619 21:16:22.530960 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.531012 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.531122 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8105728}
I20260619 21:16:22.863417 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8133a49f53cd4e0c9bcb3b4c9d5da36d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.804502 1744 rpcz_store.cc:276] 0619 21:16:19.275963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:19.282626 (+ 6663us) service_pool.cc:227] Handling call
0619 21:16:22.470615 (+3187989us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5c00096f0c. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f71a2405361 00007f71a259f1f9 00007f71a3a14441 00007f71a3a143a3 00007f71a39ff80b 00007f71a39ff2a2 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772
0619 21:16:22.470875 (+ 260us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3941929216}
I20260619 21:16:22.799957 1810 rpcz_store.cc:276] 0619 21:16:21.083993 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.132650 (+ 48657us) service_pool.cc:227] Handling call
0619 21:16:21.235443 (+102793us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b5800004598. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719b9daf11 00007f719bb5e851 00007f719bb5e702 00007f71a4a4eb09 00007f71a398a730 00007f71a3a5079b 00007f71a3a50821 00007f71a3a142b0 00007f71a39ff7f9 00007f71a39ff2a2 00007f719d0e5684
0619 21:16:21.235686 (+ 243us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:21.251205 (+ 15519us) raft_consensus.cc:1520] Early marking committed up to index 0
0619 21:16:21.251238 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0619 21:16:21.251280 (+ 42us) raft_consensus.cc:1631] Marking committed up to 3
0619 21:16:21.251309 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:22.473240 (+1221931us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b540004f954. stack: 00007f719bb64033 00007f719bb63f30 00007f719b4c2534 00007f71a51ce8b4 00007f71a51ce849 00007f719ea43471 00007f719eabef51 00007f719eabd99e 00007f71a39ff2e9 00007f719d0e5684 00007f719d0e5612 00007f719d0e5541 00007f719d0e54bc 00007f719d0e4772 00007f719cee894c 00007f719cee7d86
0619 21:16:22.473302 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0619 21:16:22.473461 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":43635968}
I20260619 21:16:22.662293 2672 raft_consensus.cc:3032] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
W20260619 21:16:22.533861 3184 log.cc:924] Time spent T b249d6b40f6943e3bb7235b7df99e1b1 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
W20260619 21:16:22.464156 1709 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.725s user 0.016s sys 0.016s
I20260619 21:16:22.871953 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
W20260619 21:16:22.872350 1709 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.389s user 0.031s sys 0.026s
I20260619 21:16:22.874231 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b249d6b40f6943e3bb7235b7df99e1b1" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.874662 1710 leader_election.cc:302] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:22.873206 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.810649 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfef072e2311405996f05452211cddca" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:22.526019 3153 consensus_meta.cc:287] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.733s user 0.009s sys 0.014s
I20260619 21:16:22.876048 1812 raft_consensus.cc:2454] T dfef072e2311405996f05452211cddca P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 6.
I20260619 21:16:22.858255 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.859756 1818 raft_consensus.cc:2454] T efba5ed7ac544242953fd0c25465b088 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
W20260619 21:16:22.527329 1653 consensus_meta.cc:287] T 331f6581b0ac4c3bbab86ecc34ececfb P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.242s user 0.028s sys 0.037s
I20260619 21:16:22.866323 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb9b500bf9a24148b2fbcd762b549521" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
W20260619 21:16:22.596724 3135 log.cc:924] Time spent T 2cb9bc6419d14ea8b9834eea6fafc54a P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 1.980s user 0.008s sys 0.006s
W20260619 21:16:22.441450 3181 log.cc:924] Time spent T 6a49a9579b0a40fdbfe77c1e282f1800 P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.152s user 0.000s sys 0.001s
I20260619 21:16:22.870844 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1270002c3c45988c3ac4fbeafaf019" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.663969 3117 leader_election.cc:288] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:22.479552 3120 leader_election.cc:288] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:22.534137 2920 consensus_meta.cc:287] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 0.575s user 0.015s sys 0.018s
I20260619 21:16:22.902837 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:22.861191 1634 connection.cc:232] RPC call timeout handler was delayed by -1.0630224979999525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260619 21:16:22.903993 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef644e16cf5b46b68fde33b180432159" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.673589 2923 raft_consensus.cc:3032] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Stepping down as leader of term 4
I20260619 21:16:22.830704 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f756a9c8b10a4992950255915336bb2c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.674710 3176 leader_election.cc:288] T 84b6bbbb60204347824250a8d71e6d3c P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:22.791237 3107 raft_consensus.cc:3037] T b249d6b40f6943e3bb7235b7df99e1b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Advancing to term 2
I20260619 21:16:22.960528 1738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.915509 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc672e451a9f4f6eb43430477e3acd1b" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:22.921509 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae81d060b714ae98341d6b5e2669f3f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.881462 1740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aedd754fcc34eecbb29d5cccbb9153a" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.945083 1742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.961683 1738 raft_consensus.cc:2396] T 24c647ebb0b64c158458c654b2462eeb P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260619 21:16:22.948184 1818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 704) took 1018 ms. Trace:
I20260619 21:16:22.962064 1818 rpcz_store.cc:276] 0619 21:16:21.929717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:21.932505 (+ 2788us) service_pool.cc:227] Handling call
0619 21:16:22.920207 (+987702us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:22.962582 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.962878 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.963387 1818 raft_consensus.cc:2454] T ccfc3736d63b4f80ab5f68b26ab104a0 P b2e7f02778ec43bdbca03dd013687c76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 6.
I20260619 21:16:22.525267 3147 leader_election.cc:288] T 4b1e21185de74e3da01e18ff99572a08 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:22.874738 3029 consensus_meta.cc:287] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.378s user 0.015s sys 0.042s
I20260619 21:16:22.865267 3104 raft_consensus.cc:482] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b2e7f02778ec43bdbca03dd013687c76)
I20260619 21:16:22.873549 3076 raft_consensus.cc:482] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
W20260619 21:16:22.951903 1634 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 46958us
I20260619 21:16:22.950641 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b2adcb38244b60bdd7ae5d5642f641" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.953437 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.953833 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:22.979741 1663 raft_consensus.cc:3037] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Advancing to term 7
I20260619 21:16:22.949266 2923 raft_consensus.cc:729] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:22.953251 3186 raft_consensus.cc:482] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:22.950193 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1857ce01be64d68891414a298ac40af" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.956866 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c655fb144a4c11b8902e23b4fac063" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:22.958514 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcc16880ccb4a85a786728439b64cc0" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.015537 1735 raft_consensus.cc:2454] T 84c655fb144a4c11b8902e23b4fac063 P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 4.
I20260619 21:16:22.947654 1666 raft_consensus.cc:2454] T 6333f19ae36b48459bbfa0490f45d128 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:23.017035 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e50dfc7ff65482ab6eb4b18b5ae06bb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:23.017683 1666 raft_consensus.cc:2454] T 5e50dfc7ff65482ab6eb4b18b5ae06bb P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:23.018580 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:22.902523 1665 raft_consensus.cc:2454] T d80394dabe604f8898e7b24149fc624f P 884c214bb0a7455d91701ec388635e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 1.
I20260619 21:16:22.964056 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21131e98f1f94e8cac1998e6c32aa089" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:22.876467 2860 consensus_meta.cc:287] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 0.881s user 0.002s sys 0.031s
W20260619 21:16:22.757215 1554 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 453967us
I20260619 21:16:22.869240 2672 raft_consensus.cc:729] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:22.969640 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980ccca53c3424db1585f12669d6d58" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.950364 3162 consensus_queue.cc:1046] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260619 21:16:22.436235 3136 raft_consensus.cc:504] T f0b6191588f14043884eceae0b64c41c P d53146739e3240239c0a3d5ea90e07fe [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.974051 1809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cb98bb4e2642acb6448e5f5ccdb1ff" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.897274 3183 raft_consensus.cc:482] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 21:16:22.906890 3126 leader_election.cc:288] T a192c7be8b2c49bda61f7d5588b3d246 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:22.975612 1816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d7929378ea14d5fbd2b7c633537d332" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
W20260619 21:16:22.956190 2942 consensus_meta.cc:287] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.941s user 0.013s sys 0.062s
I20260619 21:16:22.980250 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea6228e49c44b9889aa709158ef557" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:22.979234 1669 raft_consensus.cc:2454] T bfe558fbf8e24ddfbe844d69487cce1d P 884c214bb0a7455d91701ec388635e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 6.
I20260619 21:16:23.053489 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:23.053787 1634 leader_election.cc:334] T 5ae5f9625019495d85bc13c82d7f621b P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.629s (SENT)
I20260619 21:16:22.983404 3147 raft_consensus.cc:2726] T c28152c520fe41e5be2905e6e3b67b8a P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:23.054217 1669 raft_consensus.cc:2454] T 32ea84a8ad12422f8ec0ede64598fa44 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:23.055071 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a49a9579b0a40fdbfe77c1e282f1800" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
W20260619 21:16:23.055447 1634 leader_election.cc:334] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Timed out: RequestConsensusVote RPC to 127.0.99.2:42475 timed out after 1.858s (SENT)
I20260619 21:16:22.867179 3042 ts_tablet_manager.cc:1918] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 3.1
I20260619 21:16:22.966073 2851 leader_election.cc:288] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:23.020510 1666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce345f630e142ac9300688d473743af" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:23.024583 3049 consensus_queue.cc:1035] T 6a49a9579b0a40fdbfe77c1e282f1800 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260619 21:16:23.025251 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1e5b0f2736497bb5ea639a3071f8a2" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.057997 1813 raft_consensus.cc:2379] T cd1e5b0f2736497bb5ea639a3071f8a2 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 5: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:22.951680 3153 raft_consensus.cc:504] T 6984e19be5654fef830977bff30f643e P d53146739e3240239c0a3d5ea90e07fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.026188 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74"
I20260619 21:16:22.951416 2920 raft_consensus.cc:504] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:22.983858 1732 consensus_meta.cc:287] T a7a69751ebea42d5950577710f9b1301 P d53146739e3240239c0a3d5ea90e07fe: Time spent flushing consensus metadata: real 1.502s user 0.038s sys 0.020s
I20260619 21:16:23.029910 1660 raft_consensus.cc:2454] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 3.
I20260619 21:16:23.031371 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccfc3736d63b4f80ab5f68b26ab104a0" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:23.030368 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:23.064000 1739 raft_consensus.cc:2379] T 8ed3117ca10a4413872e39fb4535aec7 P d53146739e3240239c0a3d5ea90e07fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate d53146739e3240239c0a3d5ea90e07fe in this term.
I20260619 21:16:22.963968 3120 consensus_queue.cc:1035] T ccfc3736d63b4f80ab5f68b26ab104a0 P 884c214bb0a7455d91701ec388635e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260619 21:16:22.781234 1726 rpcz_store.cc:276] 0619 21:16:17.987842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:17.989245 (+ 1403us) service_pool.cc:227] Handling call
0619 21:16:18.010583 (+ 21338us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0619 21:16:18.033122 (+ 22539us) ts_tablet_manager.cc:676] Creating new metadata...
0619 21:16:21.317413 (+3284291us) tablet_metadata.cc:641] Metadata flushed
0619 21:16:22.208444 (+891031us) tablet_replica.cc:205] Creating consensus instance
0619 21:16:22.295853 (+ 87409us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:22.954550 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d45d883f7fa4dd285976666f28c0770" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.055859 1634 leader_election.cc:302] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 884c214bb0a7455d91701ec388635e74; no voters: b2e7f02778ec43bdbca03dd013687c76, d53146739e3240239c0a3d5ea90e07fe
W20260619 21:16:23.032325 2984 consensus_meta.cc:287] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.108s user 0.017s sys 0.020s
I20260619 21:16:22.977376 3029 raft_consensus.cc:504] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.072885 3144 raft_consensus.cc:482] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d53146739e3240239c0a3d5ea90e07fe)
I20260619 21:16:23.073874 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32ea84a8ad12422f8ec0ede64598fa44" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.085098 2672 consensus_queue.cc:260] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.088214 1669 raft_consensus.cc:1226] T 8ed3117ca10a4413872e39fb4535aec7 P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Rejecting Update request from peer b2e7f02778ec43bdbca03dd013687c76 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260619 21:16:23.089818 2977 consensus_meta.cc:287] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.296s user 0.013s sys 0.009s
I20260619 21:16:23.094796 2920 leader_election.cc:288] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 7 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
I20260619 21:16:23.096190 1732 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:35686 (request call id 76) took 5106 ms. Trace:
I20260619 21:16:22.980927 3104 raft_consensus.cc:504] T 246c3d015ec242eca8bfaf5a076692b1 P d53146739e3240239c0a3d5ea90e07fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:22.987342 3076 raft_consensus.cc:504] T 8e9a51e66e0646eeb7161c0cb670ae77 P b2e7f02778ec43bdbca03dd013687c76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:23.064849 2734 consensus_meta.cc:287] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.382s user 0.001s sys 0.019s
I20260619 21:16:23.113791 1743 raft_consensus.cc:2354] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P d53146739e3240239c0a3d5ea90e07fe [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 6. Current term is 7.
I20260619 21:16:23.061233 3042 log.cc:1196] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Deleting WAL directory at /tmp/dist-test-taskaZkkRX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781903674708472-396-0/minicluster-data/ts-0-root/wals/1764e19eb06140f8bbdb1c3aab252d9a
I20260619 21:16:23.064575 3159 raft_consensus.cc:3032] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Stepping down as leader of term 2
I20260619 21:16:23.127419 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.134248 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a192c7be8b2c49bda61f7d5588b3d246" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:23.142153 1810 raft_consensus.cc:2454] T 24c647ebb0b64c158458c654b2462eeb P b2e7f02778ec43bdbca03dd013687c76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53146739e3240239c0a3d5ea90e07fe in term 4.
I20260619 21:16:23.142267 1739 raft_consensus.cc:2454] T a192c7be8b2c49bda61f7d5588b3d246 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884c214bb0a7455d91701ec388635e74 in term 5.
I20260619 21:16:23.134907 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692d38d1f19c468eae7077dde218a62f" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.134974 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c06c6619d04c58a5ed3f65b1c82f70" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76" is_pre_election: true
I20260619 21:16:23.136332 1817 raft_consensus.cc:2379] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:23.070363 1653 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38114 (request call id 75) took 4256 ms. Trace:
I20260619 21:16:23.034895 2860 raft_consensus.cc:504] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:23.134517 2900 consensus_meta.cc:287] T 84119be03678450983c7d5a686bd3c86 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.295s user 0.000s sys 0.018s
W20260619 21:16:23.138672 2980 consensus_meta.cc:287] T 6984e19be5654fef830977bff30f643e P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.437s user 0.026s sys 0.017s
I20260619 21:16:23.112655 3138 raft_consensus.cc:2726] T 8e9a51e66e0646eeb7161c0cb670ae77 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260619 21:16:23.143862 1813 raft_consensus.cc:2379] T 25c06c6619d04c58a5ed3f65b1c82f70 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d53146739e3240239c0a3d5ea90e07fe in current term 7: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:23.162564 3029 leader_election.cc:288] T 2aedd754fcc34eecbb29d5cccbb9153a P b2e7f02778ec43bdbca03dd013687c76 [CANDIDATE]: Term 3 election: Requested vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475)
I20260619 21:16:23.138334 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c4f216cdd7450b9659feeea5fd2bf9" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:23.164004 1669 raft_consensus.cc:2454] T f9c4f216cdd7450b9659feeea5fd2bf9 P 884c214bb0a7455d91701ec388635e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 5.
I20260619 21:16:23.034294 3186 raft_consensus.cc:504] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
W20260619 21:16:23.140206 1710 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.99.3:45591, user_credentials={real_user=slave}} blocked reactor thread for 265591us
I20260619 21:16:23.060535 2942 raft_consensus.cc:504] T 6333f19ae36b48459bbfa0490f45d128 P d53146739e3240239c0a3d5ea90e07fe [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.157296 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe558fbf8e24ddfbe844d69487cce1d" candidate_uuid: "884c214bb0a7455d91701ec388635e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:23.165479 1710 leader_election.cc:302] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d53146739e3240239c0a3d5ea90e07fe; no voters: 884c214bb0a7455d91701ec388635e74, b2e7f02778ec43bdbca03dd013687c76
I20260619 21:16:23.171186 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd2b54df9ba4dac85c2eb1387dd6f03" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe"
I20260619 21:16:23.172394 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b6191588f14043884eceae0b64c41c" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:23.172698 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21131e98f1f94e8cac1998e6c32aa089" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:23.137969 2672 raft_consensus.cc:3037] T 4d7929378ea14d5fbd2b7c633537d332 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Advancing to term 3
I20260619 21:16:23.115975 2984 raft_consensus.cc:504] T e3b2adcb38244b60bdd7ae5d5642f641 P 884c214bb0a7455d91701ec388635e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.141810 3159 raft_consensus.cc:729] T 8ed3117ca10a4413872e39fb4535aec7 P b2e7f02778ec43bdbca03dd013687c76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2e7f02778ec43bdbca03dd013687c76, State: Running, Role: LEADER
I20260619 21:16:23.125859 3144 raft_consensus.cc:504] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P b2e7f02778ec43bdbca03dd013687c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.129243 2977 raft_consensus.cc:504] T bb9b500bf9a24148b2fbcd762b549521 P b2e7f02778ec43bdbca03dd013687c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.120959 2923 consensus_queue.cc:260] T deaff01bc5d54f808990dbcbaf46c0b1 P b2e7f02778ec43bdbca03dd013687c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.057720 3183 raft_consensus.cc:504] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884c214bb0a7455d91701ec388635e74" member_type: VOTER last_known_addr { host: "127.0.99.1" port: 45637 } } peers { permanent_uuid: "d53146739e3240239c0a3d5ea90e07fe" member_type: VOTER last_known_addr { host: "127.0.99.2" port: 42475 } } peers { permanent_uuid: "b2e7f02778ec43bdbca03dd013687c76" member_type: VOTER last_known_addr { host: "127.0.99.3" port: 45591 } }
I20260619 21:16:23.144970 3045 raft_consensus.cc:2229] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 [term 3 LEADER]: Raft consensus shutting down.
I20260619 21:16:23.180737 1814 raft_consensus.cc:2379] T 516fa114155140f6b508f311c4971c08 P b2e7f02778ec43bdbca03dd013687c76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 in current term 3: Already voted for candidate b2e7f02778ec43bdbca03dd013687c76 in this term.
I20260619 21:16:23.325157 1669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24c647ebb0b64c158458c654b2462eeb" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "884c214bb0a7455d91701ec388635e74" is_pre_election: true
I20260619 21:16:23.334149 1860 tablet_bootstrap.cc:654] T 6ae495cfaaca419c8611fe233762a02a P b2e7f02778ec43bdbca03dd013687c76: Neither blocks nor log segments found. Creating new log.
W20260619 21:16:23.204983 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 300297us
I20260619 21:16:23.244899 1812 raft_consensus.cc:1226] T d3404667de13461aa4b3e79e8d6a64ee P b2e7f02778ec43bdbca03dd013687c76 [term 5 LEADER]: Rejecting Update request from peer d53146739e3240239c0a3d5ea90e07fe for earlier term 4. Current term is 5. Ops: []
I20260619 21:16:23.376483 1739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28152c520fe41e5be2905e6e3b67b8a" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
I20260619 21:16:23.166795 1663 raft_consensus.cc:2454] T 25c06c6619d04c58a5ed3f65b1c82f70 P 884c214bb0a7455d91701ec388635e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e7f02778ec43bdbca03dd013687c76 in term 7.
I20260619 21:16:23.235741 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed3117ca10a4413872e39fb4535aec7" candidate_uuid: "d53146739e3240239c0a3d5ea90e07fe" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b2e7f02778ec43bdbca03dd013687c76"
I20260619 21:16:23.273501 1666 raft_consensus.cc:3037] T 5ce345f630e142ac9300688d473743af P 884c214bb0a7455d91701ec388635e74 [term 3 FOLLOWER]: Advancing to term 4
I20260619 21:16:23.309394 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80394dabe604f8898e7b24149fc624f" candidate_uuid: "b2e7f02778ec43bdbca03dd013687c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d53146739e3240239c0a3d5ea90e07fe" is_pre_election: true
@ 0x7f719849bc87 __libc_start_main at ??:0
I20260619 21:16:23.878053 1810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44478 (request call id 692) took 1076 ms. Trace:
I20260619 21:16:23.882779 1810 rpcz_store.cc:276] 0619 21:16:22.802185 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:23.150978 (+348793us) service_pool.cc:227] Handling call
0619 21:16:23.877957 (+726979us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260619 21:16:23.900991 1812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44478 (request call id 683) took 1546 ms. Trace:
I20260619 21:16:23.901489 1812 rpcz_store.cc:276] 0619 21:16:22.354476 (+ 0us) service_pool.cc:170] Inserting onto call queue
0619 21:16:23.164622 (+810146us) service_pool.cc:227] Handling call
0619 21:16:23.244704 (+ 80082us) raft_consensus.cc:1457] Updating replica for 0 ops
0619 21:16:23.900585 (+655881us) raft_consensus.cc:1683] Filling consensus response to leader.
0619 21:16:23.900890 (+ 305us) inbound_call.cc:173] Queueing success response
Metrics: {}
@ 0x294389 (unknown) at ??:?
W20260619 21:16:23.179860 3154 consensus_meta.cc:287] T e3b2adcb38244b60bdd7ae5d5642f641 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.619s user 0.022s sys 0.017s
W20260619 21:16:23.341243 3133 consensus_meta.cc:287] T cfa69d158f2b471e8638824d3d9c757b P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.631s user 0.023s sys 0.005s
W20260619 21:16:23.350190 3130 consensus_meta.cc:287] T 84119be03678450983c7d5a686bd3c86 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.418s user 0.020s sys 0.011s
W20260619 21:16:23.548509 2684 consensus_meta.cc:287] T 8f709f3b1e904d1ba8bf5592d77cc8d6 P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 1.603s user 0.023s sys 0.024s
W20260619 21:16:23.342216 3145 consensus_meta.cc:287] T 34c673e979964335b28709efdae7ece1 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.389s user 0.010s sys 0.011s
I20260619 21:16:23.152475 2734 raft_consensus.cc:2726] T 1fd2b54df9ba4dac85c2eb1387dd6f03 P 884c214bb0a7455d91701ec388635e74 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591) with higher term. Message: Invalid argument: T 1fd2b54df9ba4dac85c2eb1387dd6f03 P b2e7f02778ec43bdbca03dd013687c76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 884c214bb0a7455d91701ec388635e74 for earlier term 6. Current term is 7.
W20260619 21:16:23.400147 1783 consensus_peers.cc:487] T 1764e19eb06140f8bbdb1c3aab252d9a P b2e7f02778ec43bdbca03dd013687c76 -> Peer d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475): Couldn't send request to peer d53146739e3240239c0a3d5ea90e07fe. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260619 21:16:24.049657 1783 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.2:42475, user_credentials={real_user=slave}} blocked reactor thread for 649583us
W20260619 21:16:23.340314 3074 consensus_meta.cc:287] T 0dea6228e49c44b9889aa709158ef557 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.410s user 0.003s sys 0.030s
I20260619 21:16:23.331938 3042 ts_tablet_manager.cc:1939] T 1764e19eb06140f8bbdb1c3aab252d9a P 884c214bb0a7455d91701ec388635e74: Deleting consensus metadata
W20260619 21:16:23.164755 3180 log.cc:924] Time spent T fc672e451a9f4f6eb43430477e3acd1b P b2e7f02778ec43bdbca03dd013687c76: Append to log took a long time: real 0.700s user 0.004s sys 0.006s
I20260619 21:16:23.305238 3177 leader_election.cc:288] T d3404667de13461aa4b3e79e8d6a64ee P 884c214bb0a7455d91701ec388635e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d53146739e3240239c0a3d5ea90e07fe (127.0.99.2:42475), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)
W20260619 21:16:23.589681 3103 consensus_meta.cc:287] T 044f6297187147aba0e5ec306cc3840a P 884c214bb0a7455d91701ec388635e74: Time spent flushing consensus metadata: real 2.092s user 0.043s sys 0.021s
I20260619 21:16:23.377297 3114 raft_consensus.cc:2726] T f756a9c8b10a4992950255915336bb2c P d53146739e3240239c0a3d5ea90e07fe [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260619 21:16:23.575369 3079 consensus_meta.cc:287] T 13c1f262670941baa9933b28f3c67fc2 P b2e7f02778ec43bdbca03dd013687c76: Time spent flushing consensus metadata: real 1.854s user 0.016s sys 0.017s
I20260619 21:16:23.986058 3183 leader_election.cc:288] T 225b6f9e181a43de8dceeb75cbb1c01d P d53146739e3240239c0a3d5ea90e07fe [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 884c214bb0a7455d91701ec388635e74 (127.0.99.1:45637), b2e7f02778ec43bdbca03dd013687c76 (127.0.99.3:45591)